The State Services Commission claims that agencies’ OIA performance has ‘lifted significantly’. What does analysis of the published statistics tell us?

Summary

This turned into a longer piece than originally intended, partly because I have tried to explain things instead of assuming the reader knows all the details. For those who don’t have the time or inclination to read the whole thing here it is in three bullet points:

• The State Services Commission (SSC) has led a programme of work to ‘improve government agency practices around requests for official information under the Official Information Act.’ This programme of work is set out in a commitment made by the government in its 2016-18 Open Government Partnership National Action Plan.
• SSC’s claim that there has been a significant improvement in agencies’ performance on Official Information Act (OIA) requests is based almost entirely on figures that appear to show that 95.2% of OIA requests are responded to within the legal time limit, up from 87.6% in 2016/17.
• While any improvement in timeliness is welcome, the SSC’s claim is undermined by the fact that number of complaints to the Ombudsman about agencies extending the time limit to respond to an OIA request has increased by 150%.

Introduction

One of the perennial concerns about the operation of freedom of information (FOI) laws around the world is the timeliness of responses to requests. A phrase often repeated by requesters and regulators is that ‘information delayed is information denied’, since information released too late to be of use in a political process is just as effective in frustrating a requester as an outright refusal.

In the USA, the National Security Archive’s 2006 audit of the 10 Oldest FOI Requests in the Federal Government found that one had not been responded to in 17 years, and that multiple agencies had requests on hand that were more than 15 years old. In the UK, the Campaign for Freedom of Information called on the Information Commissioner to clamp down on delays by government departments, and used official statistics to show that 78 requests to the Treasury took more than 120 days to answer, and that several other departments had also taken more than 120 days (4 months) to respond to multiple requests.

Here in New Zealand, there are several design features in the Official Information Act 1982 (the OIA) which help to avoid the scale of the delays faced in the USA and UK. Nevertheless, in recent years a fair amount of news coverage has been dedicated to the issue, and successive Ombudsmen have made efforts to persuade agencies of the importance of timely responses. Following the publication in December 2015 of the previous Chief Ombudsman’s report, Not a game of hide and seek, the last Government decided that an appropriate way to address the issue was through the National Action Plan (NAP) that New Zealand has to produce every two years as part of its membership of the international Open Government Partnership (OGP). In the 2016-18 NAP, the government made a commitment to ‘improve government agency practices around requests for official information under the Official Information Act.’

Countries belonging to the OGP are subject to three key requirements:

• they must co-create a National Action Plan with the public every two years, setting out the concrete actions that will be taken over the following two years to make the country more open in the interests of delivering better public services, improving accountability and strengthening integrity;
• they must assess their own progress in delivering the NAP at the halfway stage and also at the end of each two-year period, and
• they must have their progress independently assessed by the country researcher employed by the OGP’s Independent Reporting Mechanism (IRM), who produces a mid-term and an end-of-term report on development of the NAP and delivery of the commitments contained in the plan.

New Zealand has just completed its second NAP, and both the government and the IRM researcher are in the process of producing their end-of-term reports. In light of this, it may be useful to look at what the government says has been achieved in relation to its commitment to ‘improve government agency practices’ in response to OIA requests.

What the government committed itself to

The 2016-18 NAP contained seven commitments. Commitment 2 was entitled ‘Improving official information practices’. The government said its objective was ‘To make government information more accessible by adopting a consistent set of agency practices in response to requests for official information.’

The NAP did not specify any outcome indicators against which this objective could be assessed. Instead, it offered what it called ‘verifiable and measurable milestones to fulfil the commitment’. These are detailed in the table below:

Verifiable and measurable milestones to fulfil the commitment	Start date	End date
Ensure information about the OIA (how to make requests, etc) and responses to requests are easy to access on agency websites. This could include development of single OIA web pages for agencies	November 2016	March 2017
Publish OIA statistics (how many requests, time taken to respond, etc)	November 2016	March 2017
Develop a clear statement of government policy on proactive release of Cabinet papers and related material	October 2016	June 2017
Develop a suite of consistent measures about OIA performance	November 2016	November 2017
Improve access to official information by publishing responses to requests on government websites and developing principles for more proactive release	November 2016	February 2017
Agencies will be supported to deliver through the development of appropriate guidance and training	November 2016	June 2018

We could question why it took until 2016, and a major report from the Ombudsman, for these basic activities to be initiated, but that’s a story – or more likely a thesis – for another day. As it is, these targets are pretty basic output indicators, mostly binary (achieved/not achieved), that have very few attributes that would enable us to assess the quality and utility of what has been delivered against what was promised.

After some prompting by the Expert Advisory Panel appointed by the government to advise it on delivery of the National Action Plan, the SSC started producing quarterly reports on what progress had been made in delivering the seven commitments.

The rest of this post examines the claims made by the SSC in its Year End (March – June 2018) report on delivering Commitment 2.[1] Instead of commenting on the activities and outputs listed under the heading ‘What have we been doing?’, I’m going to take a close look at SSC’s (and thereby the government’s) claims under another heading, ‘How did this commitment contribute to open government?’

The Commission states that the programme of work it began in late 2016 ‘has been critical in driving up system performance’ in relation to ‘agencies’ compliance with the letter and spirit of the Official Information Act’.

The assessment continues, ‘System performance has lifted significantly over the past twenty months.’  The metrics for this claim are not made clear, but later in the document some ‘Key performance statistics’ presented, and these are analysed below. Fundamentally though, SSC’s claims for improvements in ‘system performance’ appear boil down to:

• ‘an increase in engagement from agencies across the State service’; and
• an improvement in the proportion of OIA requests responded to on time.

The meaning of the first claim is unclear, although a subsequent sentence states that there has been ‘a steady increase in participation’ in the OIA Forum (a meeting of officials responsible for managing OIA requests in their agencies) since January 2017.  No OIA Forum meeting appears to have been held so far in 2018: as of 3 September 2018 the last published Forum details are for 23 November 2017. This suggests that either SSC is not good at proactive publication (which is one of the things it is supposed to be advising other agencies on how to do better), or that SSC’s claim for an increase in ‘engagement’ rests on an increase in participation only in 2017.  No data is provided to explain what this increase has been. It might have been an increase from the 40 officials that attended in May 2017, but we don’t know, because attendance numbers have not been released for any meeting except those for May 2017 – and that was in response to an OIA request.

The SSC’s claim of an increase in ‘engagement’ as a contribution to open government certainly should not be taken to mean that agencies have been increasing their engagement with requesters.  If that were the case it would be reasonable to expect some quantitative information to be provided about the number of times agencies contacted requesters to fulfil their duties under section 13 of the OIA to provide assistance where a request is unclear.  No such quantitative data is provided, and is not reported (and possibly not collected) by the SSC’s statistics on OIA requests.[2]

So how did the government do in relation to the second of SSC’s claims – improving the timeliness of responses?

SSC’s end of second year report notes that the statistics on OIA requests that it has published since January 2017 cover the 110 agencies that make up the State Services.  It claims that ‘Chief Executives are using these statistics to drive performance in their agencies.’

The report then states:

Key performance statistics include:

• The volume of OIA requests recorded for each of the three periods measured has increased. For the most recent half-year period from July to December 2017, agencies received a total of 21,323 requests, a 5% increase since 1 July 2015.
• Despite the increased workload, agencies have improved the time taken to respond. 95.2% of requests between July and December 2017 were handled within the legislated time limit. This has improved from 87.6% in 2016/17.

Workload

In relation to the first of these statistics – the increase in the number of OIA requests recorded by the agencies – this simply provides a backdrop against which the second set of figures can be contrasted.  An increase in the number of OIA requests recorded by agencies does not, in itself, tell us much, other than the workload may be increasing.[3]  But it does not tell us if the requests became more onerous to respond to, or were more specifically targeted thanks to a greater amount of information being published proactively.  It certainly does not tell us if fewer requests were being refused and more information was being released – surely the key outcome measure relevant to a programme of work the purpose of which, according to the SSC, is to ‘make government information more accessible’.

It is worth noting that the second period in question (July-December 2017) covered the run-up to, and aftermath of, the 2017 General Election.  It might reasonably be expected that General Elections would stimulate an increase in requests, but SSC has not published half yearly statistics for 2015/16 and 2016/17.[4]  This makes it impossible to compare the July-December 2017 figures for requests received against earlier periods.  However, the Ombudsman’s detailed (agency by agency) statistics on OIA complaints received are published in half-yearly periods.[5]  Publication of these only started from July 2016 onwards, but if we compare July-December 2016 to the second half of 2017, we can see that there was a 25% increase in the number of OIA complaints received (up from 538 to 673).

This is not a ringing endorsement for the SSC’s work leading to greater openness, but there are two reasons why we should be cautious about reading too much into these figures. First, unlike the UK,[6] SSC does not collect or publish data on the result of requests (refused/partially refused/released in full), so the increase in complaints might be (a) proportionate to an increase in the number of requests agencies received, and (b) due to increased public awareness of the Ombudsman and a greater willingness to make complaints, following publicity about success in reducing the backlog of old complaints. Second, we also need to keep in mind that only 2.7 percent of OIA requests result in a complaint to the Ombudsman.[7]  In the absence of data on the percentage of requests refused in full or part, it is impossible to know whether this reflects a high level of openness, people giving up on the process by not complaining to the Ombudsman, or some other reason.

With these caveats in mind, and allowing that only one year of comparable detailed data has been released by the Ombudsman and SSC since publication started in January 2017, if we compare the SSC data to the figures found in the Ombudsman’s annual reports we see the following:[8]

Table 1 – OIA request and complaint numbers

Period	Number of OIA requests to agencies	y/y % change	Number of OIA complaints to the Ombudsman	y/y % change
1 July 2015 – 30 June 2016	40,273	n/a	1,100	n/a
1 July 2016 – 30 June 2017	42,008	+4.31	1,174	+6.73
Total change from first to last period	+1,735	+4.31	+74	+6.73

Since there has been a greater percentage increase in the number of complaints to the Ombudsman than there has been in the number of requests to agencies for which SSC collates data, this also does not tend to support a claim that SSC has been successful in making government information more accessible.  A further caveat to these figures should be borne in mind though: New Zealand’s second OGP NAP covers the period October 2016 to June 2018, so until the SSC and Ombudsman publish data for the whole of the 2017/18 reporting year, and the details for the first half of 2018, we cannot know whether this situation has changed. However, if the SSC’s work has been successful, the data which is due to be published later this month should show a decrease in the overall number of OIA complaints to the Ombudsman.

Timeliness

Since the SSC do not collect or publish data on the outcomes of OIA requests, it is the second of the statistics quoted by SSC (above) upon which they base their claim for their OGP commitment activity leading to an improvement in agency performance – an increase in the proportion of requests responded to within the legislated time limit.

SSC says its figures show an increase from 87.6% of requests being responded to within the legislated time limit in 2016/17 to 95.2% in the period July-December 2017.

Before accepting this as an indication of success, it is important to understand what the Act itself requires and permits in terms of the time limit for responding to a request, and therefore what lies behind the phrase ‘handled within the legislated time limit’.

Section 15(1) of the OIA requires that agencies make a decision on the request ‘as soon as reasonably practicable, and in any case no later than 20 working days after the day on which the request is received’.  However, this outer time limit of 20 working days may be extended, under section 15A of the OIA, for two administrative reasons, relating to the quantity of information sought or the necessity to consult people in order to make a proper response to the request.[9]

So, although the OIA is clear that the ‘letter and spirit’ of the law (as SSC put it) requires agencies to make and communicate their decisions on requests ‘as soon as reasonably practicable’, the law permits agencies to take up to 20 working days ordinarily, and a longer period if they claim that one of the two administrative grounds specified for extending the time limit applies.

It is also worth noting that the time limits for making and communicating a decision on a request should not be understood to mean that this is when the requester will receive any information the agency decides to disclose.  Although the two events are frequently simultaneous (the agency includes the released information in the letter communicating its decision on the request), the Ombudsmen have interpreted section 28(5) of the OIA to mean that communicating the decision on a request, and providing the information to be released, are two separate and severable actions.  They advise agencies about this in their guidance:

If a decision is made to release information, then the agency must not unduly delay in making it available. In some cases, the agency may be justified in providing the information to the requester at a later date, after the decision is made.

Agencies should note that the distinction between the time requirements for making and communicating a decision on a request, and for making information available, can be important, especially when responding to large requests. Ideally, where a request is granted, the decision and the information will be sent to the requester together. If the information is ready to be released at that stage, not to send it would amount to ‘undue delay’. However, that may not always be possible and sometimes an agency may need extra time to prepare the information for release.[10]

The SSC statistics on the OIA do not report the actual number of working days it took for an agency to respond to an OIA request, nor the average (whether mean, mode or median).  They also do not report the number of occasions on which an agency made use of the extension provisions in section 15A.

Therefore, the SSC statistics cannot help us understand if agencies genuinely improved their performance by responding to OIA requests sooner than before, or whether they have simply started using the extension provisions more than they did before; in both cases we would see an increase in the number of requests ‘handled within the legislated time limit’.

Furthermore, as the statistics also do not report the number of occasions on which the decision on a request was communicated to the requester sooner than the information was provided (i.e. the agency made use of the permitted ability to separate the two actions), we cannot know whether requesters are receiving the information any sooner than before.  It appears some agencies are increasing the frequency with which they separate the two actions, leading to poorer outcomes for requesters: in the case of the Ministry of Social Development, more than a third of its responses adopted this practice, according to research by Mark Hanna.

What do the Ombudsman’s complaint statistics suggest might be occurring? If agencies were responding to requests sooner, and providing the information at the same time, we might expect to see:

1. A decrease in the number of complaints about a decision not being made ‘as soon as reasonably practicable’;
2. A decrease in the number of complaints about ‘delay deemed refusals’ (where an agency has not responded within the first 20 working days, or has exceeded the extended time limit it set for itself);
3. A decrease in the number of complaints about extensions of the time limit to respond; and
4. A decrease in the number of complaints about an ‘undue delay’ in supplying the information to the requester separately from the decision on the request.

In fact, the statistics from the Ombudsman show the following:

Table 2 – Types of timeliness complaints to the Ombudsman

Period	Decision not ASARP	% change	Delay deemed refusals	% change	Extensions	% change	Undue delay in release	% change
July-Dec 2016	4	n/a	107	n/a	20	n/a	5	n/a
Jan-June 2017	9	+125	108	+0.93	39	+95	14	+180
July-Dec 2017	9	0	114	+5.56	50	+28.21	11	-21.43
Total change July-Dec 2016 to July-Dec 2017	+5	+125	+7	+6.54	+30	+150	+6	+120

There has been a 125% increase in the number of complaints that agencies have not taken a decision on the request ‘as soon as reasonably practicable’ (ASARP), i.e. before the first 20 working days have elapsed.

There has been a 6.54% increase in the number of complaints about ‘delay deemed refusals’.  This means the requester believes the agency has failed to make a decision and communicate it to the requester either after 20 working days, or – if they have extended the time limit – within the extended time limit.

There has been a 150% increase in the number of complaints to the Ombudsman about agencies’ decisions to extend the time limit for making a decision on a request.

There has also been a 120% increase in the number of complaints that agencies have delayed releasing information after telling the requester they would do so.

It is unlikely that all these complaints will have been upheld by the Ombudsman, particularly as he has changed his approach, and will now ‘discontinue’ his investigations even in cases where the agency admits it breached the time limit. But if we compare the data for ‘complaints received’ to the SSC’s claim for an increase in the proportion of requests responded to within the legislated time limit, it does not tell a story of success for agencies.  Instead, it suggests that they have simply become more adept at using the extension provisions, and in separating the provision of information from communicating the decision to the requester.

Proportionally

What does the data tell us if we factor in the increase in the number of requests received?  Has there been an increase in different kinds of timeliness complaints as a proportion of requests?

Given the data available, we have to compare the whole of the 2016/17 reporting year to the first half of the 2017/18 reporting year.  But since we are looking at relative proportions, this should not be an issue.

The table below sets out the figures for requests and complaints as a proportion of requests for the same period. It then shows the percentage change in different types of timeliness complaints

Table 3 – Changes in timeliness complaints as a proportion of requests

Period	# of OIA requests	# not ASARP	% not ASARP	# DDR	% DDR	# Extensions	% Extensions	# Delayed release	% Delayed release
1 July 2016 – 30 June 2017	42,008	13	0.03	215	0.51	59	0.14	19	0.05
1 July 2017 – 31 December 2017	21,232	9	0.04	114	0.54	50	0.24	11	0.05
% change			+33.33		+5.88		+71.43		0

Conclusions

Although we should bear in mind that we are only comparing two periods and the absolute number of complaints of some types is small, there do seem to be some conclusions we can draw.

First, in spite of SSC’s claims for an increase in the percentage of decisions being made and communicated within the legislated time limit, the Ombudsman has seen a 5.88% increase in the number of ‘delay deemed refusal’ complaints, and a 33.33% increase in the number of complaints about decisions not being made ‘as soon as reasonably practicable’.

Second, there has been a 71.43% increase in the number of complaints about agencies extending the time limit for responding to a request.

Therefore, while agencies have increased the proportion of requests dealt with inside the legislatively permitted time limit (from 87.6% to 95.2%) it is questionable whether this has resulted in people receiving responses quicker. It may just be that agencies have become more adept at using the extension provision of the OIA.

Whether these extensions of the time limits have been justified is another matter, which requires review of the Ombudsman’s published statistics on the outcomes of complaints (made more difficult by the decision to record even agreed delays as ‘discontinued’).

Further data and analysis of OIA delay issues can be found in Mark Hanna’s well-researched post on the timeliness of agencies’ OIA responses, and their use of extension provisions.

Solutions

How might changes in practice or to the Official Information Act improve things?

Changes in practice are likely to arise from the availability of better information about processing of requests.  At the moment the period from the time a request is received by an agency to the time it communicates its decision to the requester is almost a black hole – the only information we have is whether the agency sought to clarify the request within the first seven working days, transferred it to another agency within the first ten working days, or extended the period for making its decision (or transferring the request) within the first twenty working days. We have no published data on the time it takes an agency to:

• Log a request in its management system
• Allocate the request to a team or specific official that knows about the subject matter of the request
• Decide how it is going to interpret the request, or seek clarification from the requester
• Start searching for the requested information within the agency’s record keeping and information management systems
• Collate the information and start considering whether any (and if so, which) withholding grounds or other provisions might provide good reason to refuse to disclose some or all of the information
• Decide whether anyone within or outside the agency needs to be consulted before a decision can be made
• When the consultation begins and ends
• When they seek the advice of the agency’s legal advisers on application of any withholding or refusal grounds
• When they start producing the copies of the information to send to the requester
• When the draft response is sent to the authorised decision maker to be signed out
• When the approved decision is communicated to the requester

In Not a game of hide and seek, the former Chief Ombudsman made a number of recommendations to agencies about collecting data to enable better informed management of systems and processes.  The government said in its OGP commitment to improve OIA practices that it would ‘Develop a suite of consistent measures about OIA performance’. The OIA statistics currently published do not provide detail on these processing steps. However, the good news is that the SSC guidance to agencies on OIA statistics does include a section on statistics for performance monitoring, including the following statement:[11]

What follows is a discussion of a wide range of statistics an agency could collect regarding their OIA system. It is a starting point to encourage each agency to:

• reflect upon its OIA handling as a system: its characteristics and behaviour as well as performance;
• consider the requesters’ experience of that system;
• extend the range of OIA statistics monitored by the agency beyond those related to compliance with the Act and those reported centrally by SSC.

With these factors in mind, agencies are encouraged to determine which statistics will be of most benefit for them to collect and monitor.

This ‘encouragement’ from the SSC is welcome.  However, at the moment we do not know if any agencies are planning to follow this advice.  Interested readers may want to make OIA requests to find out.  As the ‘owners’ of improvements in agencies’ OIA practices, we might hope that SSC will start to publish information about the uptake of their own advice by agencies.  We can also hope that the Ombudsman’s continuing programme of reviews of agencies’ OIA practices will shed light on this issue.  But before then, one of the key ways in which to drive collection of this data by agencies is for the Ombudsman to require agencies to provide him with it whenever he receives a complaint about an OIA response.  Some of this information is likely to be sought in the case of ‘delay deemed refusal’ complaints already, to support the Ombudsman’s approach to delay complaints.[12]  But making this a systematic set of information that is required by the Ombudsman of agencies in relation to all OIA investigations will help to incentivise agencies to implement the data collection recommended by the SSC.  Agencies (or even the Ombudsman) might consider this onerous or disproportionate, but they need to bear in mind that the OIA exists to serve the public, not them, and that if they invest in systems and processes to collect and publish this data, the work to collect it in individual cases will be much reduced.

These changes to administrative and management practice should however be bolstered by amendments to the OIA itself.  The Act should be amended to require agencies to capture specified processing data (of the kind listed above and in SSC’s guidance). In addition, the amendments suggested by the New Zealand Council for Civil Liberties (NZCCL) in its recent paper on improving the OIA, should be considered:

1. Limit the maximum time for any extension to 20 working days, and require the extension notice to specify what work is required to justify the extension [i.e. who needs to be consulted, or why the volume of information requested is proving difficult to search for or collate].
2. Reduce the time limit for an agency to notify the requester that an extension will be made from 20 working days to ‘as soon as possible and no later than 5 working days after the day on which the request was received’.
3. Amend the OIA to require that if a decision to release information is given to a requester, the information must be provided at the same time, not at a later date.
4. Where any information is withheld under section 18(d) because it is about to be published, this refusal ground is only available when the information will be published within four weeks of the decision on the OIA request.
5. Impose a per-day financial penalty on agencies for delays in response beyond these limits.

Requiring agencies to include in their annual reports how much they had to pay in fines for late responses would assist Parliament, the media and the public to hold them to account.  The fines could be paid to the requester, or into a dedicated fund for investigative journalism managed by a body like New Zealand on Air.  They could even be paid to Archives New Zealand to fund its work of auditing agencies information management systems – another under-examined aspect of the operation of the OIA that would bear further scrutiny.

Finally though, a word of caution.  While this piece began by noting the problems caused by delays in responding to requests for information, we also need consider the consequences of concentrating on the timeliness of responses as opposed to the quality of them.  The current focus on the speed of agencies’ responses (and Ombudsman investigations) risks incentivising making quick decisions, rather than correct decisions.  If you’re an agency that knows it will be ‘named and shamed’ for the timeliness of its OIA responses, are you going to concentrate on the correct application of withholding grounds and the public interest test, or on getting any decision, however likely it is to be rejected by the requester or an Ombudsman, out of the door on time?

---

Footnotes
[1]    Commitment 2 year end report: March – June 2018, accessed from http://www.ogp.org.nz/assets/Uploads/ogp-2337733.pdf on 1 August 2018

[2]    SSC’s statistics on agencies’ processing of OIA requests can be downloaded from: http://ssc.govt.nz/official-information-statistics

[3]    Indeed, it may only tell us that agencies have improved the recording of the requests they have received following criticisms made in Not a game of hide and seek.

[4]   In addition, analysis of the figures from Ombudsman Annual Reports between 2001/2 and 2016/17 suggests little correlation between the occurrence of a general election and the number of OIA complaints received. (Increased number of complaints in 2002/3 and 2011/12, but decreases in 2005/6, 2008/9 and 2014/15.)

[5]    The Ombudsman’s OIA complaint statistics can be downloaded from http://www.ombudsman.parliament.nz/resources-and-publications/oia-complaints-data

[6]    https://www.gov.uk/government/collections/government-foi-statistics

[7]    However, there has been an increase from 2.73% to 2.79%.

[8]    The Ombudsman’s annual reports can be downloaded from http://www.ombudsman.parliament.nz/resources-and-publications/corporate-documents/annual-reports

[9]   Section 15A(1) defines the two possible grounds for extending the time limit as:

• the request is for a large quantity of official information or necessitates a search through a large quantity of information and meeting the original time limit would unreasonably interfere with the operations of the department or the Minister of the Crown or the organisation; or
• consultations necessary to make a decision on the request are such that a proper response to the request cannot reasonably be made within the original time limit.

[10]   The OIA for Ministers and agencies, Office of the Ombudsman, Wellington, June 2016. Accessed on 3 August 2018 from http://www.ombudsman.parliament.nz/system/paperclip/document_files/document_files/2457/original/the_oia_for_agencies.pdf?1516146865

[11]   Selection and Reporting of Official Information Act Statistics, State Services Commission, accessed on 4/9/18 from https://ssc.govt.nz/sites/all/files/Selection-Reporting-of-OIA-Statistics-v2.pdf

[12]   The Ombudsman’s approach to delay complaints, Office of the Ombudsman, June 2017, accessed on 4/9/18 from http://www.ombudsman.parliament.nz/ckeditor_assets/attachments/494/ombudsmen_s_approach_to_delay_complaints.pdf