Is every internet based census doomed to #censusfail?

by | Aug 25, 2016

Watching #CensusFail unfold over the last couple of weeks was easily as gripping as some of the photo-finish Olympic events in Rio, August 9th arrived with great anticipation yet only a fraction of Australian’s could complete their forms online on census day as required. The Australian Bureau of Statistics (ABS) appeared to have taken all the right steps – they bought a system from the biggest vendor around (what’s that adage about never getting fired for buying from ….), did performance load testing, put some geocaching in place to limit denial of service attacks from outside Australia and other measures along these lines.
Sadly something (as yet undisclosed) went wrong, the website crashed within minutes and 23M Australian’s couldn’t log on to complete their compulsory census form. The situation deteriorated with call centres overloaded, citizens worried about fines for non-completion, the ABS’s scheduled social media continuing as though nothing had happened inciting ridicule, officials denial of issues followed by a blame game – have all turned this into a bit of a circus.
So you can imagine my surprise when in the middle of #CensusFail the New Zealand government agency responsible for the Census, Statistics New Zealand, announced it’s intention to buy an internet based collection system – this appeared on the government tender system before the ABS site was even back up and operating:

We are seeking to appoint a supplier or suppliers to deliver the services required to enable the successful delivery of the modernised 2018 Census operation. More specifically, to deliver the services associated with the Contact Centre and Internet Collection System components of the 2018 Census Collection System, including those needed to ensure the seamless integration between systems that is crucial to improving New Zealanders’ experience of the census. (The Internet Collection System is the primary tool that New Zealanders will use to record their census information, and the Contact Centre is the primary way for them to request paper forms and access codes, and an essential source of any help if required.)

Timing is everything and for me, this timing will forever link our future NZ electronic census with Australia’s experience.

The implications of #Censusfail for Australia and others

Trust – or lack of – is the primary issue Australia and any other countries like my own adopting an electronic census face. It is unclear whether the ABS elicited requirements from citizens or consulted in an inclusive manner reading the reports, it appears they took a risk based purchasing approach vs collaborative consultative one.
Even before the failure last week Australian citizens reported data privacy concerns primarily due to the need for every user to be authenticated and the retention period of name and address information stored from 18 months to 4 years. This generated social media activity by some publicly stating they will not complete the electronic census opting to fill in the traditional paper form with no identifying name or address information filled in.
Post #CensusFail the appearance hackers were involved, or the potential for future hacking to occur, have further compounded these data retention concerns. Couple this with the perception the ABS have misled citizens during the two days their site was down and the ongoing lack of disclosure as to what caused the outage is fueling further mistrust. The upshot here is the ABS have a long haul to entice citizens back online to complete their forms, or to call overloaded call centres to request paper copies, all within the legislative time frame and to a critical minimum participating number.
On the positive front for the Australian government, it does seem people have moved on social media activity wise already:

Yes there are internet based census success stories

Estonia and Canada are two great examples of countries who have successfully implemented internet-based census solutions, both achieved positive engagement with the online census and have carried out more than one round now, this article provides a reasonable overview of the positive citizen engagement:

In January 2012, Statistics Estonia held the first e-census in the country. During one month, Estonians managed to break a world record – 66% of the population participated in the census over the internet, beating Canada’s previous record of 54.6% from 2011.

A raft of countries have deployed electronic voting machines and internet connected voting machines with varying technologies employed for citizen validation from retina scanning to barcoded single-use cards. Estonia is also leading the world in internet based voting at a national level, Canada has deployed at local government level with some cities and states deploying internet-based voting. Limited Cantons in Switzerland, one state in India and even Australia have limited pilot groups in internet based voting trials – often for remote citizens such as armed forces abroad during elections.
From the research I have done it does surprise me that governments are not necessarily focusing on the challenge of access for all citizens to undertake an online census or vote, this requires high-speed internet for all, a device and based on our own nation’s challenges – lifting the digital literacy of every citizen. The primary challenges nations seem to be tackling are the validation of a unique individual, security (transport, data, cyber) and overcoming trust issues, will my government act prudently with my data? It certainly feels like this census challenge comes down to understanding requirements and user experience. Just to prove it’s not a tech challenge success stories like this have emerged too – 2 students who built an online Census site during a hackathon weekend, very cool.
Questions are – Are we ready for this move online yet? (Estonia appears to be) and what can the Australian case study teach us all about what not to do?
Citynews certainly sum up the possible challenges of internet based solutions before the consumer is ready. Enjoy Vic.
Victoria MacLennan is a reformed techo from the data and information management world – who now focuses on creating jobs and opportunities.
She is passionate about Open Data, Data Privacy and Governance so will blog on those topics occasionally. 
You can read What do President Obama, the inventor of the Internet and Open Data have in common? or all of Victoria’s blogs here.
We run regular Defining Data Requirments courses to bridge the technical-business divide when gathering data requirements, find out more here.

  1. Nancy Linton (Senior Adviser, Communications, 2018 Census)

    As you say in your recent blog about the latest Australian census, timing is everything, and so we thought we’d better correct you on your assertion that we released a GETS tender for our New Zealand census collection system during the Australian Census in early August 2016.
    In fact, we actually asked for requests for proposals for our internet collection system in August 2015, and we are now well underway building and testing our new system for the 2018 Census.
    We’re running a comprehensive testing programme to help us make sure that the system we’re building will be responsive and secure. We will, of course, be providing paper forms for anyone who would prefer them as well.
    We also have a long tradition of working closely with census colleagues in Canada, the UK, the USA and Australia to learn from them and share best practice.
    With around 18 months to go until our next census, we’ll be making sure that we’re listening, observing and refining our own approach, so that everyone in New Zealand can participate and help refresh the data that is used across New Zealand to inform decision making and plan services – services that matter to us all.
    (Nancy Linton, on behalf of Statistics New Zealand)

    • Victoria Maclennan

      Thanks for the update Nancy. I quite incorrectly accepted this information sent to me at face value as it was presented to me – as you point out the dates are 2015, it’s uncanny timing. Will look forward to a successful NZ outcome unlike the Australian experience. Apologies for this mistake.

  2. Jquery Chat Script

    I don’t think so this actually dooms as even my friend owns an internet broad band so now even I wish to know more of it and this information seems to me to be a bit abrupt and no more longing to be the way it used to work.

Leave a Reply to Jquery Chat Script Cancel reply

Your email address will not be published. Required fields are marked *