It is a great honour for me to be invited to give the Johnian lecture, and to do so during the celebrations of the College’s 500th year. And it is a great pleasure to be doing so under the Presidency of Mike Brearley, who has been my friend since we first met during the scholarship examination at St. John’s in 1959.

I have always held the Johnian lecture in some awe: in fact since its inception, in 1978, when it was given by my then chief, Sir Douglas Wass. Douglas, who was then the Treasury Permanent Secretary, gave a typically penetrating and elegant critique of monetarism, which was at the time the orthodoxy of the Conservative Opposition – and, as he has since told me, he was never forgiven, by Ministers in Margaret Thatcher’s administration, for so doing. So the Johnian lecture got off to a cracking start, and I see a daunting line of predecessors at this lectern.

I came up to St. John’s to read classics in 1960, and was taught by John Crook, Guy Lee, Renford Bambrough and R.L. Howland: a formidable group, as were my contemporaries, who included Malcolm Schofield and Michael Silk as well as Mike Brearley. After Part I of the Classical Tripos I was persuaded, mostly by Mike Brearley, to switch, as he was doing, to Part II of the Moral Sciences Tripos: a move which led to my embarking upon an academic career as a philosopher, beginning with a Research Fellowship at St. John’s. That career, a few years later, I reluctantly abandoned, in order to pursue the career which I then felt – and still feel – to be a high calling : to be a civil servant, with deep involvement in the business of governing the country. I did not want to be a politician, at the top of the government tree. I had a vision of rationality in government, of knowledge and analysis, and I felt that, to be a civil servant, to be required to offer advice, information and help to those elected by the British people to govern them, who, as politicians, were necessarily bound by the constraints of democratic politics; to administer and implement the resultant policies ; all of this, I felt, would be difficult and satisfying; and so it was. I was, of course, wholly unsure whether my abilities and my skills would fit me for this rôle.

This is a question on which I have often pondered. Was a classical and philosophical education a suitable foundation for my subsequent career, which at different times put me in very senior posts with responsibility for, for example, UK monetary and fiscal policy, financial regulation, tax and energy policy, the administration of British science, trade and industry and much besides?

What I have come to realise is that, when you are at or near the top of any big institution – and, most particularly, near the top of government – you need a huge range of skills and experience – a range far beyond the capability of any mortal to acquire in one lifetime. As a senior Treasury official you should, ideally, be an expert in macro- and micro- economics, a statistician, an accountant, a lawyer; someone expert in constitutional and Parliamentary theory and practice, in management, behavioural psychology, and leadership; never mind the expertise you should have in the actual subject-matter of government with which you are at different times concerned: health, education, transport, foreign affairs, defence and so on.

My time at St. John’s gave me absolutely none of this expertise. What did it give me? First and foremost, intellectual self-confidence: the appetite to confront an entirely new field, a new set of problems, to read and listen to the principal sources to which we were guided by our supervisors, to work out what we thought, then to discuss it with these teachers on the basis of pretended or sometimes actual equality, and also to discuss these issues with our contemporaries. We were engaged in the pursuit of knowledge, in analysis, in enquiry, and in rationality. A close attention to argument was required, to the nuances of meaning, to words – but not, alas, to numbers, the signal omission in my preparation for later things.

These values of rationality and enquiry are, I believe, core characteristics of St. John’s, as they are of another St. John’s College, St. John the Baptist in Oxford University, whose President I have had the honour to be over the past ten years; and they are the core characteristics of many other colleges and universities. These values are not, in the main, under attack at the present time, and for that we should be very grateful. The attack on these institutions is on other grounds: principally on the question of fees, who will finance higher education, the students, the State, or philanthropists, and the implications of that decision; on the range of subjects which may be studied in universities; and on the institutions’ freedom to select those as students they believe most capable of profiting from the education they offer. These are serious attacks, which could be the subject-matter of another lecture on another day. But, let us record, with gratitude, there is in our country at present little or no attempt to interfere with the processes of individual academic disciplines, or to influence the conclusions they reach.

In government the situation is different. There are strong forces at work whose natural outcome is, I suggest, to demote rationality, analysis and the pursuit of knowledge within government.

There has been much discussion of these forces, and of their origins: the sound-bite, the three bullet-point interview, the need to respond instantly to every event which may be used to criticise government, preferably with action or with a new initiative; then there is, more generally, the large and growing power of the new information media, and the challenge to, and weakness of, all authority.

At the heart of all of this there is a deep flaw in democratic government. Ministers, naturally, want to be elected, re-elected, and their rivals want them to be thrown out. Ministers need to persuade the electorate of the rightness of their actions, and of their power to right the wrongs which afflict us, wrongs which are constantly paraded by their rivals as evidence of Ministers’ unfitness for office. A powerful alliance springs up between Ministers pursuing those objectives and those in the Press and media who wish to make money and themselves exercise power. This alliance is, I believe, the greatest source of corruption in modern times. It dwarfs the petty corruption revealed in the Parliamentary expenses scandal. It is the genus of which the Murdoch affair is a species.

When I was a junior Treasury official I once worked for an austere and withdrawn undersecretary. When, during the 1976 economic crisis, we together went through our client spending Department’s budget, looking for savings, he struck out all government advertising. I thought this was a bit much – but he said, with passion, that it was a form of corruption, to spend money raised, in effect, by force, through taxation, to persuade the electorate that their money was well spent, as a prelude to raising more money from them by taxation.

In the face of a hypercritical stream of comment, 24 hours a day, 7 days in every week, broadcast to a public with, in general, little respect for any person or body which dares to assume a rôle of authority, Ministers and their advisers strenuously seek to present their case as persuasively as possible. It is entirely right that they should do so. But it is not right if they make deals with individual journalists or editors or proprietors – to provide, say, news now in exchange for later favours, for sympathetic treatment at some later date.

Nor is it right if they cross the line from persuasive presentation of their policies and actions to the manipulation of information, or to interference with the publication of Departmental information and statistics.

In Whitehall these developments have led to a hyper-sensitivity to the media and to outside commentary and criticism, and to the huge growth in influence first, in the 1980s, of Departmental Press Offices, then of Special Advisers; then to the growing influence of a new kind of Departmental Minister whose consuming interest is in what the next day’s Press will say – or, if he has a longer time-horizon, what the weekend’s Press will say.

A growth in such influences means, necessarily, a reduction in other influences. In Whitehall it has meant a diminishing interest in analysis and enquiry, and, in the field of government information, a growing interest in the persuasive Press release, with its careful selection of facts and numbers, designed to communicate as effectively as possible some predetermined message.

Others have charted this familiar history in greater detail and depth than I can do today. But it brings me to the substance on which I want to speak in this lecture, and also to a point in my personal history. It was six years since I had retired from the civil service when, in 2007, I was asked if I would be interested in becoming Chair of the new Statistics Authority, which was then the subject of a Bill before Parliament. I was interested, because it seemed to me that the Statistics and Registration Service Act 2007, as the Bill subsequently became, offered an unparalleled opportunity to arrest, or to push back, the mischief which I so much deplored: the manipulation of this highly important species of government information – official statistics – for political ends. Here was an opportunity to strengthen the forces of rationality in government, which have always been, and are, there, but are in recent times on the defensive: the forces for objectivity, dispassionate analysis, impartiality and honesty.

The Statistics Act, and the Authority it created, have certainly improved the environment in which this critical component of government information operates. The Statistics Authority is, by statute, an independent body, reporting directly to Parliament and not reporting to Parliament through Ministers. I am, as is a senior judge or a Minister, a Crown appointment; not, as Permanent Secretaries are, appointed by Ministers. The Authority has, by statute, two tasks: one, to be the governing body of the UK’s National Statistics Institute, the Office for National Statistics (ONS); and, two, to be the regulatory body of all official statistics whatever their source. On the first of these, as governing body of the ONS, we are there to give the public assurance that these vital statistics – on GDP, unemployment, prices, balance of payments, immigration and population statistics – are produced and published, without any involvement of politicians, by professional statisticians working to the highest standards of which they are capable. On the second of our roles, as regulator of all official or government statistics, we have the opportunity, or rather the duty, to ensure that all such statistics are produced and published without political interference, and to the highest professional standards. 

Let me give you a few examples of how we have, I believe, been effective in these two roles over the past four years.

First, an example of us in our role as governing body of the Office for National Statistics. In March 2009 the ONS published a statistical release on the country of birth and nationality of workers in employment which provided evidence for the view that a large proportion of the new jobs created since 1997 had been filled by foreign people, or by people born abroad. The ONS was immediately subjected to a fierce political attack, in which it was suggested that their analysis, which undermined one important element in the then Prime Minister’s political platform, was politically motivated. It was not politically motivated; the Authority said, in a high-profile public statement, that it was not; the Authority was believed; and the storm blew over. That analysis, which was published by the ONS to assist informed public debate in an area of great political controversy, would, I can assure you, not have been published under the pre-Statistics Act regime.

Now, four years after the Statistics Act received Royal Assent people are beginning to get used to the fact that the ONS is wholly independent of government: they witness them providing every day statistics which are often deeply uncomfortable for the government ; and which are, equally, often very supportive of the government’s policies and claims. People see that they are statistics which may be relied upon to be objective, impartial and honest. That is quite an achievement.

My second example – or, rather, examples, as I shall give two – are of the Statistics Authority in our second rôle, as the regulator of all official statistics. In December 2008 the then Prime Minister’s office arranged a photo-opportunity to demonstrate how well the government’s policies in combatting knife crime were working. There were interviews with the families of knife-crime victims, with celebrities and so on. Some statistics were paraded around the Newsrooms of Fleet Street, which purported to show a reduction in knife-attack hospital admissions in the areas in which the government’s initiative had been introduced.

The professional statisticians in the NHS immediately protested, to the head of their profession and to the National Statistician, that their advice about these statistics had been ignored by the Prime Minister’s office: the statistics were not ready for publication, were unchecked, and incomplete. I then wrote to the Prime Minister’s office, and said, in a letter which I sent to Parliament and placed on our website, that the Prime Minister’s use of these statistics had been irregular, premature, selective, and deeply corrosive of public trust in official statistics. A political and media storm followed. The Prime Minister’s Office, and the Home Secretary, made public apologies for their misuse of these statistics, and the Cabinet Secretary later issued instructions to every official and Special Adviser in Whitehall, that the Code of Practice for Official Statistics, which clearly outlawed such practices, must always be followed, and that the last word on statistical matters must always be with the Department’s professional statisticians. This was a major break-through. There is a large and growing volume of evidence that these radically new instructions, repeated at the beginning of the Coalition Government, and addressed also to incoming Ministers, have been very influential and effective.

My second example of our regulatory function concerns an intervention which involved the other side of the party divide. Just before the 2010 Election campaign the Conservative Shadow Home Secretary, Mr. Chris Grayling, said on radio that violent crime had doubled under the Labour Government, and that the Home Office’s statistics showed this to be so. The radio interviewer objected that the Home Office’s published statistics made it clear that police recorded crime had undergone important definitional changes during the period in question, and that straightforward comparisons between the figures on either side of the definitional changes could not be made. The interviewer also pointed out that, on the other measure of crime published by the Home Office, the British Crime Survey, violent crime had fallen appreciably during the Labour years in government. Mr. Grayling repeatedly replied that the Home Office’s figures were the Home Office’s figures, and that they simply showed that violent crime had doubled.

I then thought it proper to write an open letter to Mr. Grayling, setting out the complex statistical picture on violent crime. I said that if he continued to say what he had said he risked misleading the British public on the matter. Again, a media and political storm blew up, and the Opposition, under parliamentary and media pressure, modified the line on violent crime they had been advancing, both at the national level and in many local newspapers and constituencies.

These are just three examples which I have selected of the Statistics Act at work but you will see from them several of the limitations of the Act. One is that we, the Statistics Authority, have very weak powers. We cannot, as the Financial Services Authority for example can, fine people. All we can do is to write a critical letter and publicise it. That, you may say, is enough. But, please note, repetition dulls our effectiveness: I had better not write too often. And I must not, in order to achieve the necessary publicity for our work, myself begin to form an improper relationship with the Press, of which I have already complained: so I have no Press Office machine, nor will I have one.

The second limitation is that our field of operations is, by statute, limited to official statistics. So for me to criticise the Opposition was unexpected: I did so on the basis that the Opposition’s use of official statistics in that instance misrepresented official statistics in a way damaging to their integrity, and to public trust in them; a slim, but I think, a sound basis for my intervention.

One measure of the effectiveness of the Statistics Act has been the continuing support the Authority has had over the past four years from those in Parliament who see it as their role to scrutinise the work of the Executive – notably the Public Administration Select Committee and its successive Chairs. They have strongly supported our efforts to strengthen the independence and objectivity of official statistics.

I am sorry to say, however, that the Coalition Government has been so far luke-warm in its support for these objectives. Before last year’s Election I wrote to the leaders of all the major parties with suggestions as to how to strengthen the independence and perceived objectivity of official statistics. I had no reply from the Labour and Liberal Democratic parties. But Mr. Cameron wrote a positive reply, and I was told by one of his senior colleagues that the Conservatives, if they won the Election, would wish to take “the high moral ground on statistics”.

Encouraged by this, I wrote again to Mr. Cameron on his first day in office as Prime Minister. I made three suggestions, none of which would involve extra expenditure or would take Parliamentary time. I asked him to strengthen the position of the civil service head of the Government Statistical Service, the National Statistician, so that she would have a stronger voice in major decisions about statistical output and staffing in Government Departments, where the control of statisticians across the Government Statistical Service had been left by the Statistics Act in the hands of Departmental Ministers and their Permanent Secretaries.

My second suggestion was that I asked the Prime Minister to arrange for the Statistics Authority to be consulted before cuts were made to statistical work in Departments, so that the longer-term interests of the statistical system, and thus of the information base for rational public policy-making, could be taken into account in making the significant cuts in statistics which were, clearly, going to be needed in the new age of austerity. And, finally, I asked the Prime Minister to bring to an end the deplorable practice, called Pre-release Access, whereby statistical releases are shown in advance to Ministers and advisers, before publication to Parliament and the public.

All three of my requests have been rejected. There is not time today to discuss the first two; and their purport is, perhaps, obvious enough. But let me say a little more about the rejection of my third request, to end Pre-release Access for Ministers and their advisers. Our present pre-release access arrangements allow UK Ministers and their advisers 24 hours notice of statistical publications, ahead of Parliament and the public. This is wholly inconsistent with UN and EU Codes of Practice, and also with best international practice. It encourages the public to think – as they do seem to think – that Ministers and their advisers routinely interfere with statistical publications before their release.

In recent polls only one in six people think that official statistics are not manipulated by Ministers; and three out of five people think there is some dishonesty about the way statistics are presented by the Government. The European Commission recently sponsored a survey of how much the public in each of the 27 member countries trust their Government’s statistics. The UK, I am sorry to say, came 27th; right at the bottom of the class.

Ending Pre-release Access would directly address this regrettable lack of trust. Pre-release Access creates possibilities for leaks and premature unauthorised release and for dishonesty; and also, where there is no dishonesty, it creates the impression that dishonesty is taking place.

When, recently, the Treasury, through nothing more than a clerical error, gave pre-release access to some market-sensitive economic statistics to around 400 unauthorised people, I wrote to George Osborne pointing out that this very unfortunate error could not have happened if he and his colleagues had stuck to their policy on this matter when they were in opposition: the Conservative Party was then firmly against pre-release access. George Osborne has since written to me, again rejecting my request for tighter rules. He says that the public expect Ministers to be briefed about their statistics when they are published, and 24 hours notice is needed for this briefing to take place. I do not find this convincing. This is an expectation which could easily be changed. It has been changed in the USA and a number of other countries. I accept however, that abolishing Pre-release Access would be inconvenient for our spin doctors, who have grown hugely in numbers in recent decades, and who, I was reliably told, briefed Ministers on this matter when they discussed my request last summer.

My time as Chair of the Statistics Authority is approaching its end. I have been privileged to have been given the opportunity, in my last public service post, to make a contribution towards strengthening the values within government which have all along inspired me, and which my education at St John’s taught me to hold dear. Objectivity, impartiality, enquiry and analysis have been valued by the civil service for many years, certainly as far back as the Northcote-Trevelyan reforms of the mid-nineteenth century. They are valued to this day in many quarters. They should be powerful characteristics of any government, in any country.

It has been my privilege today to tell you about some of the efforts which we in the Statistics Authority have been able to make, over the past four years, to consolidate and strengthen the British government’s adherence to these values. Thank you for listening to me.