LSE - Small Logo
LSE - Small Logo

Helena Vieira

June 6th, 2017

Regulating in the dark: the challenges of overseeing the secret services

0 comments | 1 shares

Estimated reading time: 5 minutes

Helena Vieira

June 6th, 2017

Regulating in the dark: the challenges of overseeing the secret services

0 comments | 1 shares

Estimated reading time: 5 minutes

One of the most well-known principles in the canon of Jeremy Bentham’s writings on government is the general principle of transparency. All activities, according to Bentham, were to be made open so as to allow for external scrutiny. One sector, however, was exempted from this universal principle: the security or intelligence services. The reason for this exemption appears straightforward; security services, by their very nature, have to operate outside the glare of public attention in order to perform their work.

At the same time, the secrecy of operations also calls for some degree of regulation and oversight; after all, discretion can be abused – the state’s covert activities to make individuals’ lives transparent require disciplining constraint. The regulation of the security state is therefore a very special, and particularly tricky case for the study of the regulation of government activities.


In an age where the threat of terrorism has, once again, become a feature of daily life, the regulation of intelligence services is also an area that has become increasingly important as different intelligence services have launched recruitment drives; as concerns about access to encrypted communication have escalated, and the world of digital technologies is said to fundamentally alter the nature of intelligence work; and as the context of and conditions for national and international co-operation have changed.

What then can be said about recent trends in regulation and oversight? This is arguably not a question for those fascinated by a James Bond-like glamorous lifestyle. This is more the world of political and public concern with agencies that possess extraordinary powers to interfere in private and personal matters, that have coercive powers, and whose main objective is the minimisation of threat to the state and its citizens. It is also a world in which different understandings regarding an individual’s right to privacy clash. Such rights have been enshrined in Article 8 of the European Convention of Human Rights – the right to privacy can only be limited by public authority, also on security grounds, in accordance with the law and as required for a democratic society.

Recent incidents include the concern with the extensive surveillance by national intelligence services on national and non-national private citizens, politicians and businesses. There have been concerns about the lopsided nature of intelligence services in observing extremist activities such as showing a remarkable negligence in monitoring right-wing extremist sympathisers.

Furthermore, the 9/11 Commission Report and other incidents have highlighted the difficulties of ensuring national, let alone international, information exchange. In other cases, there has been rather extensive collaboration as evidenced in the recent inquiries into the collaboration between the German BND and the US-NSA. This also links to examples of so-called intelligence failures, where information was detected, but not acted upon. Attempts in the US have remained fraught as individual agencies anxiously protect their turf vis-à-vis the Department of Homeland Security and other co-ordination initiatives. Pooling of expertise is emerging across the European Union (as part of the ‘Counter Terrorism Group’) after 2001, but has remained problematic given the preferences for bilateral agreements. Similar reluctance exists when it comes to national services’ willingness to supply Europol with information.

Oversight problems

Of course, problems with the (oversight of) intelligence services are far from novel – concerns with the activities of intelligence services have been a recurring feature throughout the post-1945 period, including concerns about infiltration in highest places of government (such as then West German Chancellor Willy Brandt’s special advisor, Günter Guillaume), double agents (such as the infamous ‘Cambridge Five’) and ‘cowboy’ intelligence activities in diverse parts of the world (such as Jamaica and Northern Ireland). In the US, concerns about the activities of the intelligence services led to a formalization of oversight in the 1970s.

In debates over the regulation of intelligence services, one does not have to look very far to encounter the trade-off between the functional prerequisite to operate covertly and the ‘costs’ of being accountable and transparent to a sceptical political class and the wider public. Constitutional courts, such as the German federal constitutional court, have been highly critical regarding procedural protection against the abuse of discretional powers. Courts have, therefore, become regulators in their own right.

Intelligence services are very difficult to control – neither their daily activities nor their achievements are easily observable. Only failure can be identified, and here it may have to do more with blame-avoiding behaviours of others than actual failure. There are some controls over inputs, and one may be able to assess procedural appropriateness. One traditional tool in such cases is to rely on ‘professionalization’. By careful selection and training, intelligence services are supposedly committed to constitutional values. But such a strategy is somewhat problematic in an age where the priority is to massively expand and the security state relies on ‘security cleared’ contractors. Such bureaucratic recruitment drives are marred by severe difficulties, as noted by Rizzi and Borden in this issue.

Environmental changes

The wider environment in which intelligence services operate has also changed. There have been traditional differences in official acknowledgement as for e.g. the UK intelligence services were only officially recognised in the late 1980s and early 1990s. Some intelligence services now publish their addresses, they provide some information on their websites (even pictures of their buildings), and the actual identity of their leaders is publicly known. Yet, the availability of information on budgets and staffing numbers remains less open. In contrast to the world of three or four decades ago, there has been a notable trend towards ‘voluntary accountability’ to appeal to public support and legitimacy.

This emphasis on self-presentation is mirrored by extensive changes in the wider oversight ecology. There has been an increasing reliance on internal legal clearance procedures. This has, in turn, led to a considerable growth of in-house lawyers to provide advice on the legality of particular operations. Such a growth in formal legal requirements might be interpreted as a response to (the perception of) distrusting politicians and a fear of ‘moral panic’ about revelations regarding particular operations. To some, this juridification represents a challenge to the execution of the core functions of intelligence services.

Furthermore, there has also been a rise in external watchdogs and oversight bodies. In the US, the role of the Inspector General has changed from an earlier age in which a position in that office was seen as a ‘recovery period’ from tricky intelligence operations. Instead, since 1989 when the position was placed on a statutory basis, the Inspector General has become increasingly resourceful and distant from the intelligence services.

The challenges of parliamentary oversight

In the UK, there has been a remarkable change in parliamentary oversight, partly in response to pressures from the European Convention of Human Rights. The first Intelligence and Security Committee (ISC) was a statutory, not a parliamentary committee; it reported to the prime minister, was hand-picked by the prime minister and operated in closed sessions, with its reports being prone to redactions. Requests for information could be refused on grounds of sensitivity. The Justice and Security Act 2013 made the ISC a committee of parliament with extended powers of oversight, and with members being appointed by Parliament (following nomination by the Prime Minister in consultation with the Leader of the Opposition).

Whatever the formal standing of legislative oversight committees, their actual role is problematic as committees are supposed to play a dual function in providing both support and oversight. A too critical oversight performance, one that is also linked to critical commentary in the media, is likely to lead to a breakdown in the relationship between the committee and the intelligence services. At the same time, too much ‘cheerleading’ for the intelligence services will also be seen as problematic, as is an ‘ostrich’ style oversight in which parliamentarians are seen to be avoiding any form of difficult confrontation – only to be the first to criticize intelligence services once issues have appeared in public).

Similarly, as noted by Amy Zegart (2011) in the case of the US, oversight is limited by a lack of interest by legislators (the oversight of intelligence services being unlikely to be a vote-winner in constituencies) and by lack of power over budgetary appropriations. Other observers suggest that politicians might be keen to play to the gallery of public attention in times of failure and public outcry, but they will be reluctant to engage when difficult choices are presented to them. There are also questions as to how to bring together different parliamentary overseers. The latter issue has, in the German case, led to the creation of a special Beauftragte (parliamentary commissioner) role in parliament, tasked with providing co-ordination between different parliamentary oversight bodies. Again, concerns have arisen as to the background of potential appointees; with ‘insiders’ being seen as ‘too close’, whereas outsiders are viewed as potentially ineffective due to lack of inside knowledge.

Regulatory overseers might not have the problem of limited political attention spans, although they face similar issues when it comes to questions of ‘critical distance’. Their specific challenge therefore is to highlight to the wider public that they are engaging in active and critical oversight, without necessarily revealing the extent and the content of their interactions. How, therefore, such bodies are accountable, and how they pursue strategies of engagement with interested parties (and who is regarded as a legitimate party) remains highly controversial within and across jurisdictions.

International cooperation

Oversight is also problematic when it comes to international cooperation. One country’s legal interpretations of international human rights conventions might differ from another country’s. Competing interpretations about human rights might be seen to stand in the way of effective cooperation. Indeed, such differences might also reflect different national traditions with regard to the role of legal advice; and such traditions will ultimately lead to further conflicts between the rival interpretations about human rights and demands for ‘more cooperation’.

Conflicts with technology companies

The powers of the intelligence state are not only relevant in view of their direct powers over individuals. Conflicts between technology companies and intelligence services have become particularly prominent in recent times over access demands to the information stored on smartphones and encrypted communication systems. Again, the issue of providing the state with a formal or even informal backdoor to technological systems is one that places competing claims about collective security interests against each other. These conflicts link to two fundamental debates. One is the extent to which private organizations can be forced to cooperate with the security state. Such relationships require a degree of procedural formality, even if they are secretive. The other relates to the potential differences in the snooping powers of private versus public organizations. The difference in terms of the coercive powers of the state is clearly one major difference. However, this difference should not stand in the way of critical questions with regard to the use of private data and ‘snooping’ capacities by technology firms.

The regulation of security services in an age of international cooperation and modern communication technologies is therefore one of the most vexing problems in the regulation of contemporary executive power. The tensions identified by Bentham are impossible to design away; tensions between civil liberties and security concerns, the role of competing understandings as to what counts as evidence, how to ensure the upholding of constitutional values, and how to sustain critical, but non-adversarial oversight constitute some of the most important questions facing liberal democracy in an age where fears about security are central to the political and public agenda.



  • This blog post was originally published by Risk & Regulation, the magazine of the Centre for the Analysis of Risk and Regulations. See the original version here
  • As part of the Regulation in Crisis? seminar series, CARR held a workshop on the regulation of homeland security, bringing together leading practitioners and academics.
  • The post gives the views of its authors, not the position of LSE Business Review or the London School of Economics.
  • Featured image credit: Secret intelligence services, by Peter Trimming, under a CC-BY-SA-2.0 licence
  • Before commenting, please read our Comment Policy.

Martin Lodge is director of the Centre for Analysis of Risk and Regulation (CARR) at LSE. He is also a professor of Political Science and Public Policy. His key research interests are in the areas of executive politics and regulation. He teaches on courses in Public Administration & Public Policy, Public Management, and Regulation. Professor Lodge is co-editor of the journal Public Administration and of a book series on ‘Executive Politics & Governance’ (with Palgrave). He chairs the UK Political Science Association’s specialist group on Executive Politics and Governance and co-chairs the International Political Science Association’s Structure and Organisation of Government research committee. He is part of the TransCrisis research consortium (funded under European Commission’s Horizon2020 programme).



About the author

Helena Vieira

Posted In: LSE Authors

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.