Constitutional Affairs Secretary and Lord Chancellor
Lord Falconer of Thoroton
I'm delighted to join you this morning for this seminar, organised jointly by the British Academy and The National Archives.
Let me first thank Peter Hennessy for chairing today's event. Peter's research, conducted over many, many years, has shone a light on the inner workings of government: from the mechanics of the Whitehall machine to the realities of Cold War Britain. He is quite simply without equal in his academic use of the nation's archives to research Britain's post war history.
Let me also take a moment to pay tribute to the work of Sarah Tyacke - who retired as Chief Executive of the National Archives in October last year.
Sarah led the National Archives through a period of immense organisational change, not least the transition from the Public Records Office to the National Archives and the challenges of managing and preserving information in a digital age. She transformed the UK national archives into one of the leading national archives in the world. She deserves our great thanks.
Her successor, Natalie Ceeney, is now carrying that work on and I look forward to watching the National Archives reach even greater heights under Natalie's guidance.
Today's seminar is an important and timely one. Freedom of Information is an important tool for historical research - but it is a tool that is at present under-utilised, perhaps not by the people who are here today, but by the academic community as a whole.
One of the reasons for this, is that FOI is still a relatively new regime. It, understandably, will take time to bed down within public authorities and for a settled regime to emerge..
But we should not overlook the significance of Freedom of Information. Its introduction - one year ago - was a constitutionally significant moment, comparable to reforms such as the incorporation of the European Convention on Human Rights into British law.
FOI is something that Governments and oppositions had talked about doing for many years, but this Government did it. Oppositions often forget about FOI once in government. We did not.
January 2005 marked a milestone - replacing the opaque, secret, obstructive 'need to know' with an open, transparent, statutory - and enforceable - 'right to know'.
It has been a long road to this point. For decades previously, Governments of all stripes had progressed slowly, cautiously and reluctantly.
The period from the Official Secrets Act in 1911, through the subsequent conflicts and the Cold War era, has been characterised as a time where Governments interpreted information as something that they saw as necessarily secret.
Before 1959, there was no legal right of access to public records held at the then Public Records Office. Certain discretionary practices on disclosure existed but it was not until the Public Records Act was passed in 1958 that a legal right - subject to exemptions - allowed access to public records.
That Act required courts, government departments and other public bodies to transfer records selected for permanent preservation to the archives before they reached the 30 year point. Under these arrangements, the records remained closed to the public until they were 50 years old.
The Public Records Act passed in 1967 reduced this period of closure to 30 years. I will be coming back to the vexed question of the '30 year rule' later in my speech.
But, there was little further progress towards openness until the 1980's - where a renewed focus on openness in Government was present but, critically, in the 80s, it was openness in others - such as local authorities, not in central Government itself.
Even the revised Official Secrets Act in 1989 was devised because its predecessor was unworkable - not out of any great desire for openess.
From 1994 until Freedom of Information came fully into force last year, people wanting to access information had to rely on the non-statutory Code of Practice on Access to Government Information.
A move in the right direction, yes. But far from a great leap forward.
This was an information access regime that lacked teeth. It was negotiable, it was non-statutory - and it failed to oblige public authorities to disclose information even where there was public interest in doing so. It did apply to central government departments, but failed to capture many thousands of other public bodies that hold information that is in the public interest. The Code simply did not carry the necessary clout.
So, until Freedom of Information came into force, researchers or members of the public had to rely on a complex and inadequate patchwork of information access codes - either stand-alone or built into other legislation.
FOI now means:
Let me mention briefly the impact the Act has had already - on the public and on Government itself, before turning to it's impact on historians and scholars.
At its core, FOI simply means the general public can find out more about the decisions that affect their lives.
Thousands of information requests have been answered, amounting to nearly 2,000 information releases each and every month. In many cases, this means new information, never previously released, is now in the public domain.
If you want to find out about success rates of certain operations at your local hospital - you can.
If you want to find out about hygiene standard in school kitchens - you can.
FOI is enabling public debate to be better informed and more constructive.
Equally, the obligations FOI places on public institutions will improve the quality of decision-making. FOI encourages proactive disclosure of information where possible.
This, of course, is a profound culture change. But central government and other public authorities have responded positively. In the first nine months of FOI, almost 30,000 [29,575] applications were received by central government bodies - 60% released in full, 86% released within statutory deadlines. I can tell you, from being within government, that FOI is having an impact. It is a thorough improvement, increasing the rigour of decisions.
The third area is the impact of FOI for scholars, in particular, for historians.
In countries that have had information access legislation for many years, academic use of FOI is well developed. Less so here. But I am certain that the legislation that we have introduced in this country will have considerable benefits for those engaged in academic research across a wide range of disciplines.
EH Carr considered the facts of history to be simply those which historians have selected for scrutiny. On this level, the benefits of FOI are obvious. FOI means more documents disclosed - providing more sources, sooner, for the contemporary historian to scrutinise.
I can think of few areas - indeed if there are any areas - of academic activity that are untouched by the benefits of FOI. We perhaps think first of those disciplines concerned with politics and the process of Government. But this is also a tool for economists, for historians and for scientists.
The Government, and the 100,000 public authorities covered by the Act, constitutes the most significant source of information in the country.
Consider for a moment the materials released under Freedom of Information, from the National Archives alone.
Information about the Chernobyl disaster or discussions in the 1970s of devolution in Scotland and Wales, are now in the public domain for historians, geographers, social scientists to plunder. So too are Cabinet Secretary's notebooks, the first three of which were released this January.
In all, 70% of FOI requests to the National Archives resulted in full disclosure. This changes considerably the material - in terms of the overall amount and in terms of the quality - that historians can draw from.
These documents open up new avenues: for completely new research; for the re-evaluation of existing research; for research about historical theory.
For some historians the value of the documents will be in what they tell us about the actions and behaviours of public authorities in modern times. For others it will be the very essence of what is released and why that is a source of interest.
Historical researchers have for years used government documents as the cornerstone for research. FOI provides a rich seam to be mined in helping you to identify what is - and what is not - historically significant.
But the interaction between academics and the Archives is far from a one way process. This is not about you going cap in hand completing request after request until you get the information you need.
Quite the opposite.
Freedom of Information is about more than just waiting for requests to come in and then responding. It is about pre-empting what can be disclosed - and disclosing it.
The National Archives, working with the transferring Government Departments, is pro-actively making a huge amount of information less than 30 years old available under Freedom of Information.
On the first day FOI was operational, the National Archives released around 50,000 files that were less than 30 years old - a wealth of information ranging from research at Porton Down to arranged marriages. The information released also touched on some topics that are still subject of much debate today including Home Office files on reviewing gambling laws and the classification of cannabis.
But this wasn't just a one-off. In December 2004, the National Archives held almost 300,000 closed records. Today, through a combination of pro-active release by government departments and FOI requests from the public, that number has fallen to below 200,000.
This is a significant - perhaps-unparalleled - opening of our historical archives.
But - and there is always a but - there are, of course, obvious limits to what can be released. Not least due to the resources required to physically sort through what is primarily a paper record and to assess the information for release.
FOI has brought unprecedented rights of access, but the Act also sets out the right and responsible limits. The legislation strikes the balance between release and non-disclosure, to allow public authorities to continue to operate effectively.
There will of course be occasions where information cannot be released, as exemptions under the Act apply - national security is one such example.
We have seen a vast number of requests to find out information about issues which really do add value to people's lives, or to the sum of human knowledge, or to research. And making a positive contribution to the quality, accuracy and completeness of public debate. This is exactly why we brought in this legislation.
However, it is also true, inevitably, that this culture is being undermined by requests under the Act which arguably do not impact so positively - like what a central government department spends on toilet paper or make-up, or whether written proof can be provided under the Act of a Minister's existence.
Responsible users of the Act and supporters of the legislation would surely agree that these sorts of requests are frivolous, and sometimes even vexatious, and that spending time answering them is not how public resources ought to be used.
The Information Commissioner has made his position abundantly clear. He has said that claims that are 'manifestly unreasonable' should be resisted under section 14 of the Act - claims which would serve no public good or impose substantial burdens on the financial and human resources of public authorities.
My department will be issuing guidance complementing that from the Information Commissioner, to help public authorities handle these types of requests.
I don't believe this is a concern in terms of requests from the academic community - but it is a serious issue for the operation of Freedom of Information as a whole.
In other areas, we need to look closely at the balance between disclosure and non-disclosure. Let me say a few words here about the so-called 30-year rule. This needs a little explanation.
I should actually refer to the 30-year rules, as there are two:
The confusion arises when we muddle what is reasonable administrative practice and what is disclosed or not disclosed in response to FOI requests.
Governments generate huge numbers of records - there needs to be a sensible regime in terms of their physical management and storage. Departments follow the convention applied by section 3 and keep them 'in-house' for almost 30 years, at which point they are assessed. If they are suitable for preservation - and the vast majority are not - they are moved to be stored in the Archives. 30 years is the cut off point for assessment - in many, many cases, records are transferred and made available to the public long before they reach the 30-year point.
This is plain common sense - it is sensible administration of what, across Government, is a vast number of records.
But FOI transforms the way we approach the second part of the deal. The same statutory right to know under the Freedom of Information Act applies to information whether it is 29, 30 or 31 years old.
Every body responsible for state records has to decide which records shall be kept permanently.
Each public body has to transfer those documents that are to be permanently retained to the National Archives within 30 years of the creation of the document, unless, with the approval of the Lord Chancellor, they decide for good administrative reasons they should be kept in the home department.
Once documents have been transferred to the National Archives, they must make arrangements for the public to views them. Before the FOI Act came into force, section 5 of the Public Records Act provided that the public only had a right to view them after 30 years (subject to exceptions which gave the state wide discretion to extend or reduce the time).
The provision that the public could not normally see the documents until 30 years had expired is what was traditionally called the 30 year rule.
And that rule has been repealed by the FOI Act.
Every document, whether in the National Archives or not, is now subject to the regime set out in the Act.
The National Archives apply that regime like everybody else. That regime contains provisions which relax the number of exemptions which apply after 30 years. That relaxation applies wherever the document or information is held.
The impact of this change is wholly beneficial.
The pre-Act arrangements for research were that 30 years was the norm. There were very many exceptions, often driven either by the motives of the public authority - for example they wanted a history written of the events - or the relationship between that part of government and the researcher. But these exceptions had no legal or coherent basis. There was no way in which they could be enforced. They were dependent on the goodwill of the state.
Now there are all those cases and, much more importantly, a whole legal regime which can be used to unlock archive material whether in the National Archives or not, whether 30 years old or not, if it falls to be disclosed under the Act.
This part of FOI is not developed, because it has not been tested. It will be important. It means the researcher will look to the law, not just to relationships or to what the department hands out.
It will take time to work through its full uses. It means many projects will depend on the success of FOI applications.
But it means there is no automatic need to wait 30 years. And it means we will not have to wait nearly so long for many pieces of detailed historical work.
History will be written earlier and fuller in many more cases.
Under Freedom of Information citizens everywhere have a right to access information from any time and at any time.
Requests for records that are less than 30 years old are now assessed for release under the framework of the Freedom of Information Act.
This represents a significant change to the Code of Practice on Access to Government Information. The Code provided no statutory rights - and the appeals process to the Ombudsman was significantly weaker than the powers of the Information Commissioner under FOI.
Of course, some exemptions still apply under FOI when the information is over 30 years old, such as national security.
FOI allows for this: when it comes to historical information, the passage of time sometimes alters the public interest arguments around disclosure. In some cases, the sensitivities of a document lessen over time.
In other cases, the public interest arguments against disclosure will remain strong for very many years - sometimes for 50 or 70 years. In those cases it is only right that we act responsibly to continue to withhold information where it is in the public interest to do so.
But let me be clear - the idea that you can't get information that is less than 30 years old is a red herring.
Unless exemptions apply, information should be made available to the widest possible audience at the earliest possible opportunity.
That is a pen picture of FOI and where we are now. But where do we go from here?
If you look at any other successful regimes, there is clear evidence that a constructive approach between 'user' and 'provider' is not just a good idea, but is fundamental to it's ongoing success. It builds mutual understanding and fosters a well-informed community of users. This has been crucial to the enduring success of information access legislation in many overseas jurisdictions.
This is why, next month, my colleague Cathy Ashton will chair the first meeting of a panel of Freedom of Information users. Representatives from a number of constituencies will tell us about their experiences in accessing information. This will allow us to make sure that users' views inform and shape the development of FOI policy.
We are as committed to providing a high quality service to academic users of Freedom of Information as we are to the thousands of other requesters amongst the general public.
We welcome views from any user of the legislation to suggest areas where we can do better or where there are unrealised opportunities. I understand that some historians and researchers feel that in some ways things do not work as well for them under FOI as they did before. There is no reason why this should be so - if you have suggestions for how things could work better, let us know. If it is possible to implement suggestions - we will do so.
We also need to look at the range of bodies covered by FOI.
The FOI Act currently allows for private bodies that have public functions to be covered by it.
Before looking at extending coverage of the legislation to private bodies, we needed to learn the lessons of FOI in the 100,000 public bodies currently covered in the first year. We need to allow FOI to bed in while public authorities develop an efficient way of operating under what is an entirely new regime.
We are looking right now at how exactly we will do this. The sort of private bodies we're likely to extend to are privately run prisons and school academies - which should be subject to the same principles of openness and transparency that applies to the thousands of other public authorities covered by FOI.
But we will ensure that - where we extend the scope - it does not have a disproportionate impact on the ability of those bodies to carry out their functions. We need to get the balance right.
There are other obvious challenges in the years to come.
Technology, for example. The advent of Freedom of Information has been preceded by a revolution in the way that information is recorded, searched and made available.
Desktop computers, e-mail and the internet have altered the way in which Government business is done, and information stored. Capturing and preserving the historical records presents large organisations such as Government departments with new challenges.
Archives themselves have changed from being purely a physical place - open to those who wish to visit the archives in person - to becoming a virtual place, readily searchable online and accessible to a far wider audience than was previously possible.
The combination of technology and Freedom of Information provides the amateur historian with greater access to historical records. The opening up the archives to non-specialists and amateur historians, as well as those researching in their professional capacities, is certainly an area to watch.
This is exactly what Freedom of Information is about. Legitimate access to give greater insight. If you'll forgive me, it's information for the many, not the few.
Being able to access information brings with it to power to know how decisions are and were made and the issues that are and were considered.
Greater access to official information is the route to a stronger relationship between citizen and government of the day.
And greater access to official information is also the route to a clearer understanding of our past and our shared history: the backbone of our modern, diverse society.
I hope the historians of tomorrow will look back and see that the introduction of Freedom of Information was a truly significant moment that irreversibly changed the way we govern. And changed it for the better.
But I hope the historians of today will fully embrace FOI.
As has been said, 'history will not solve the shortcomings of public policy, but public policy will always be inadequate without knowledge of history'.
Thank you for your commitment to pursue that knowledge.
I hope you enjoy the rest of the seminar.