Telecommunications data retention

From Wikipedia, the free encyclopedia

In the field of telecommunications, data retention (or data preservation) generally refers to the storage of call detail records (CDRs) of telephony and internet traffic and transaction data (IPDRs) by governments and commercial organisations. In the case of government data retention, the data that is stored is usually of telephone calls made and received, emails sent and received and web sites visited. Location data is also collected. The primary objective in government data retention is traffic analysis and mass surveillance. By analysing the retained data governments can identify the locations of individuals, an individual's associates and the members of a group such as political opponents.

In the case of commercial data retention, the data retained will usually be on transactions and web sites visited.

Data retention also covers data collected by other means (e.g. by automatic numberplate recognition systems) and held by government and commercial organisations.

See also Telecommunications data preservation.


Data retention in the European Union

On 15 March 2006 the European Union formally adopted Directive 2006/24/EC, on "the retention of data generated or processed in connection with the provision of publicly available electronic communications services or of public communications networks and amending Directive 2002/58/EC" [1][2]

The Directive requires Member States to ensure that communications providers must retain, for a period of between 6 months and 2 years, necessary data as specified in the Directive

  • to trace and identify the source of a communication;
  • to trace and identify the destination of a communication;
  • to identify the date, time and duration of a communication;
  • to identify the type of communication;
  • to identify the communication device;
  • to identify the location of mobile communication equipment.

The data is required to be available to competent national authorities in specific cases, "for the purpose of the investigation, detection and prosecution of serious crime, as defined by each Member State in its national law".

The Civil Liberties, Justice and Home Affairs (LIBE) committee of the European Parliament had recommended that data be retained for a maximum of only 12 months; that it be made available only with a judicial warrant, and only in connection with crimes serious enough to qualify for a European arrest warrant; and that communications providers be compensated for the cost of the data storage. But these recommendations were put aside after private representations were made, over the heads of the MEPs who had been specialising on the dossier, by the German government to the German leaders of the dominant Socialist and Christian Democrat blocs in the Parliament. [3].

The Directive as adopted covers fixed telephony, mobile telephony, Internet access, Internet email and Internet telephony. Member States are required to transpose it into national law within 18 months - ie no later than September 2007. However, they may if they wish postpone the application of the Directive to Internet access, Internet email and Internet telephony for a further 18 months after this date. A majority of Member States have indicated that they will indeed exercise this option.

Member States retain the flexibility to go substantially further than the Directive mandates. Subject to notification to the Commission, they may require data to be held longer than the two year maximum set by the Directive. They retain the freedom under Article 15(1) of the earlier Directive 2002/58/EC to legislate official access to the retained data for purposes beyond those set out in the Data Retention Directive. (Germany, for example, has indicated that it seeks to make retained data admissible in certain civil copyright cases [4]). And they maintain the freedom to require retention of additional data, beyond that specified by the Directive. (For example, the draft of the Danish executive order to implement the Directive [5] seeks to require ISPs to log the source, time and destination of every single internet data packet, rather than just the details of logins and logouts to the ISP that the Directive actually seems to require).

As of January 2007 there are still several months to go before the deadline for national laws to be adopted to implement the Directive. But at the time the Directive was passed there were three countries in the EU with legal data retention actually in force. Italy retains data for four years and Ireland for three years. The UK has an extensive system of data retention, under a voluntary agreement made with the industry, but it had not yet been placed on a statutory basis. Belgium had re-introduced the possibility of data retention on 13 June 2005 with a new telecommunication law, but the royal decree stipulating what kind of data should be stored, by which market parties and for what period of time was never issued.

Data retention in the United Kingdom

The United Kingdom [UK] has a system of voluntary data retention which derives from Part 11 of the Anti-Terrorism, Crime and Security Act 2001. Telephone operators and Internet Service Providers retain some data [see below] under a voluntary arrangement with the UK Home Office.

The Part 11 of the Act contains a number of sections which deal with the retention of communications data by fixed line and mobile telephone service providers and internet service providers [service providers]. Communications data includes data which identifies the users of services, data which identifies which services were used and when they were used, and data which identifies who the user contacted. It does not include the content of communications. For example, in the case of a call from a mobile telephone the data to be retained would include data identifying the owner of the phone, who was called, the duration of the call and the approximate locations of both parties. It would not include what was said during the call.

The Act requires the Secretary of State for the Home Office to issue a voluntary code of practice on data retention. This has been done. A code of practice has been issued and contains the requirements set out below. If the Secretary of State believes that the voluntary code of practice is not effective then he may make the code compulsory by issuing a statutory instrument which must be approved by both Houses of Parliament. This has not yet been done.

Other legislation affecting data retention includes the Data Protection Act 1998 and the Privacy and Electronic Communications (EC Directive) Regulations 2003. These require service providers to destroy or anonymise data when it is not longer required for commercial purposes [e.g. billing]. The provisions of Part 2 of the Anti-Terrorism, Crime & Security Act 2001 override these requirements.

Any compulsory scheme of data retention would also be affected by Article 8 (the respect for the right of privacy) of the European Convention on Human Rights. This would require the destruction or anonymisation of communications data, but article 8(2) of the European Convention on Human Rights permits an interference with individual’s right to privacy if it is necessary in the interests of national security and the prevention and detection of certain types of crime. If the UK government was not to be challenged under Article 8 of the Human Rights Act it would need to demonstrate that any statutory requirement for data retention was proportionate.

Home Office Voluntary Code of Practice on Data Retention

Subscriber Information - retention period 12 months Subscriber details relating to the person e.g. Name, date of birth, installation and billing address, payment methods, account/credit card details Contact information (information held about the subscriber but not verified by the CSP) e.g. Telephone number, email address Identity of services subscribed to (information determined by the communication service provider) Customer reference/account number, list of services subscribed to Telephony: telephone number(s), IMEI, IMSI(s) Email: email address(es), IP at registration Instant messaging: Internet Message Handle, IP at registration ISP - dial-in: Log-in, CLI at registration (if kept) ISP - always-on: Unique identifiers, MAC address (if kept), ADSL end points, IP tunnel address

Telephony Data - retention period 12 months All numbers (or other identifiers e.g. name@bt) associated with call (e.g. physical/presentational/network Assigned CLI, DNI, IMSI, IMEI, exchange/divert numbers) Date and time of start of call Duration of call/date and time of end of call Type of call (if available) Location data at start and/or end of call, in form of lat/long reference. Cell site data from time cell ceases to be used. IMSI/MSISDN/IMEI mappings. For GPRS & 3G, date and time of connection, IMSI, IP address assigned. Mobile data exchanged with foreign operators; IMSI & MSISDN, sets of GSM triples, sets of 3G quintuples, global titles of equipment communicating with or about the subscriber.

In addition it has been reported that the UK police and security services have the following additional capabilities on mobile telephones, provided by telephone operators, but not covered by the voluntary code.

  1. The capability to record to the content of selected conversations.
  2. The capability to determine the location of a mobile telephone to within a few yards by using triangulation and multiple base stations.
  3. The capability to remotely activate the microphones of some mobile telephones. The UK Financial Times of the 2nd August 2005 [page 4 of the UK edition] reported that the UK police can ask mobile phone operators to download special spying software to a mobile telephone without the user’s knowledge or permission. When this has been done the authorities can turn on the microphone of a mobile telephone and listen to any conversations in its vicinity. This capability only exists for mobile telephones which can accept downloaded software. The telephone must be turned on for the microphone to be activated, but the user does not have to be making a call.

SMS, EMS and MMS Data - retention period 6 months. Calling number, IMEI - Called number, IMEI - Date and time of sending - Delivery receipt - if available - Location data when messages sent and received, in form of lat/long reference.

Email Data – retention period 6 months. Log-on (authentication user name, date and time of log-in/log-off, IP address logged-in from) - sent email (authentication user name, from/to/cc email addresses, date and time sent) - received email (authentication user name, from/to email addresses, date and time received).

ISP Data – retention period 6 months. Log-on (authentication user name, date and time of log-in/log-off, IP address assigned, Dial-up: CLI and number dialed, Always-on: ADSL end point/MAC address (If available).

Web Activity Logs – retention period 4 days. Proxy server logs (date/time, IP address used, URL’s visited, services. The data types here will be restricted solely to Communications Data and exclude content of communication. Web browsing information is retained to the extent that only the host machine or domain name (web site name) is disclosed. For example, within a communication, data identifying would be traffic data, whereas data identifying would be content and not subject to retention.

Other Services - retention period relative to service provided. Instant Message Type Services (log-on/off time) if available.

Collateral Data - retention period relative to data to which it is related. Data needed to interpret other communications data, for example the mapping between cell mast identifiers and their location, and the translation of dialing (as supported by IN networks.

Retention of other data

Postal data - retention period unknown. Information written on the outside of a postal item (such as a letter or parcel, online tracking of postal items, records of postal items, such as records of registered, recorded or special delivery postal items, records of parcel consignment, delivery and collection

Banking data - seven years. It has been reported in the UK Economist magazine that UK banks are required to retain data on all financial transactions for seven years. This has not been verified. It is not clear whether data on credit card transactions is also retained for seven years.

Vehicle movement data - two years. Leaked documents from the Association of Chief Police Officers [ACPO] have revealed that the UK is planning to collect data from a nationwide network of automatic numberplate recognition cameras and store the data for two years in a controversial new centre being built at Hendon. This data could then be linked with other data held by the government and watchlists from the police and security services.

Access to retained data

The bodies which are able to access retained data in the United Kingdom are listed in the Regulation of Investigatory Powers Act 2000 (RIPA). These are -

  • Police forces (as defined in section 81(1) of RIPA)
  • National Criminal Intelligence Service
  • National Crime Squad
  • HM Customs and Excise
  • Inland Revenue
  • Security Service
  • Secret Intelligence Service
  • Government Communications Headquarters

Reasons for accessing retained data

The justifications for accessing retained data in the United Kingdom are set out in the Regulation of Investigatory Powers Act 2000 (RIPA). They are -

  • in the interests of national security;
  • for the purpose of preventing or detecting crime or of preventing disorder;
  • in the interests of the economic well-being of the United Kingdom;
  • in the interests of public safety;
  • for the purpose of protecting public health;
  • for the purpose of assessing or collecting any tax, duty, levy or other imposition, contribution or charge payable to a government department;
  • for the purpose, in an emergency, of preventing death or injury or any damage to a person's physical or mental health, or of mitigating any injury or damage to a person's physical or mental health;
  • for any purpose (not falling into the above) which is specified for the purposes of this subsection by an order made by the Secretary of State.

Data retention in Italy

In July 2005 new legal requirements on data retention came into force in Italy.

Subscriber information Internet cafes and public telephone shops with at least three terminals must seek a license permit within 30 days from the Ministry of Home Affairs. They must also store traffic data for a period which may be determined later by administrative decree. WIFI hotspots and locations that do not store traffic data have to secure ID information from users before allowing them to log on. For example, users may be required to enter a number from an ID card or driving license. It is not clear how this information is validated. Mobile telephony must identify them selves before service activation, or before a SIM may be obtained. Resellers of mobile subscriptions or pre-paid cards must verify the identity of purchasers and retain a photocopy of identity cards.

Telephony data Data, including location data, on fixed line and mobile telephony must be retained for 29 months. There is no requirement to store the content of calls. Telephony operators must retain a record of all unsuccessful dial attempts.

ISP data Internet service providers must retain all data for at least six months. The law does not specify exactly what traffic data must be retained. There is no requirement to store the content of internet communications.

Legality The legislation of July 2005 enables data retention by outlawing all the relevant data protection provisions until 31 December 2007. Under the data protection provisions, service providers are obliged to anonymise or delete traffic data when they no longer need it to process the communication or to send bills (with a maximum of 6 months). The traffic data which will now be retained can be used for anti-terrorism purposes and for general penal enforcement of criminal offences.

Italy already required the retention of telephony traffic data for 48 months, but without location data. Italy has adopted the EU Directive on Privacy and Electronic Communications 2002 but with an exemption to the requirement to erase traffic data.

Data retention in Denmark

Denmark has implemented the EU data retention directive and much more, by logging all internet flow or sessions between operators and operators and consumers. See full executive order here.

  1. "2.2.1. Session logging (section 5(1) of the Executive Order) Providers of access to the internet must, in respect of the initiating and terminating package of an internet session, retain data that identifies the sending and receiving internet protocol address (in the following called IP address), the sending and receiving port number and the transmission protocol."
  1. "2.2.2. Sampling (section 5(4) of the Executive Order) The obligation to retain data about the initiating and terminating package of an internet session does not apply to providers in case such retention is not technically feasible in their systems. In that case, data must instead be retained for every 500th package that is part of an end user’s communication on the internet."
  1. "2.2.5. Hot spots (section 5(3) of the Executive Order) In addition to the internet data that must otherwise be retained, the provider must retain data that identifies the precise geographic or physical location of a hot spot and the identity of the communication equipment used. This means that a provider of internet access via a hot spot must retain data on a user’s access to the internet and, at the same time, retain data that identifies the geographic location of the hot spot in question."

Commercial data retention

Amazon is known to retain extensive data on customer transactions. Google is also known to retain data on searches, and other transactions. If a company is based in the USA the FBI can obtain access to such information by means of a National Security Letter (NSL). The Electronic Frontier Foundation states that "NSLs are secret subpoenas issued directly by the FBI without any judicial oversight. These secret subpoenas allow the FBI to demand that online service providers or ecommerce companies produce records of their customers transactions. The FBI can issue NSLs for information about people who haven't committed any crimes.

NSLs are practically immune to judicial review. They are accompanied by gag orders that allow no exception for talking to lawyers and provide no effective opportunity for the recipients to challenge them in court. This secret subpoena authority, which was expanded by the controversial USA PATRIOT Act, could be applied to nearly any online service provider for practically any type of record, without a court ever knowing." The Washington Post has published a well researched article on the FBI's use of National Security Letters.

Arguments on data retention

It is often argued that data retention was necessary to combat terrorism. However, data retention cannot prevent any terrorist attacks. At best it may, as its proponents claim, assist the police to find the culprits and their accomplices after an attack has already taken place. The authorities in Spain and the United Kingdom have claimed that retained telephony data made a significant contribution to police enquires into the 11 March 2004 Madrid train bombings and the 7 July 2005 London bombings.

The opponents of data retention make the following arguments –

  1. Schemes for data retention do not make provisions for adequate regulation of the data retention process and for independent judicial oversight.
  2. Data retention is an invasion of privacy and a disproportionate response to the threat of terrorism.
  3. It is easy for terrorists to avoid having their communications recorded. The Home Office Voluntary Code of Practice of Data Retention admits that there are some internet protocols which cannot be effectively monitored. It would be possible for terrorists to avoid monitoring by using peer to peer technologies, internet cafes, anonymous proxies or several other methods. The police forces of the EU are sceptical about the value of data retention. Heinz Kiefer, president of Eurocop [European Confederation of Police] issued a press statement saying "it remains easy for criminals to avoid detection through fairly simple means, for example mobile phone cards can be purchased from foreign providers and frequently switched. The result would be that a vast effort is made with little more effect on criminals and terrorists than to slightly irritate them. Activities like these are unlikely to boost citizens’ confidence in the EU’s ability to deliver solutions to their demand for protection against serious crime and terrorism.’
  4. The hardware and software required to store all the retained data would be extremely costly. The costs of retaining data would not only fall on Internet Service Providers and telephone companies, but also on all companies and other organisations which would need to retain records of traffic passing through their switchboards and servers.
  5. Data retention gives excessive power to the state to monitor the lives of individual citizens.
  6. Data retention may be abused by the police to monitor the activities of any group which may come into conflict with the state; including ones which are engaged in legitimate protests. The UK police have used anti terrorism powers against groups opposed to the war in Iraq and protestors at an arms fair. The definition or terrorism in the UK Terrorism Act 2000 includes not only action, but the threat of action, involving serious violence against a person, or serious damage to property, for the purposes of advancing a "political, religious or ideological cause". There is concern that the definition is vaguely worded and could be applied supporters of animal liberation, anti-war demonstrators and many others.
  7. Even if data retention may be justified, the retention periods proposed in some cases are excessive. It has been argued that a period of five days for web activity logs and ninety days for all other data would be adequate for police purposes.

Protection against data retention

The current directive proposal (see above) would force ISPs to record the internet communications of its users. The basic assumption is that this information can be used to identify with whom someone, whether innocent citizen or terrorist, communicated throughout a specific timespan. Believing that such as mandate would be useful is ignoring that some very committed community of crypto professionals has been preparing for such legislation for decades. Below are some strategies available today to anyone to protect themselves, avoid such traces, and render such expensive and legally dubious logging operations useless.


Several organizations have setup VPNs so their employees can safely access the private data of the company. In this case the only data retained is the connection to the VPN server, that will contain all the data served to the EU citizen encrypted.

The connections, if the company is in the EU, will be recorded as a company access, mixed with all the other employees access. To single out any user, the company would have to be tapped and that the citizen would have to use it after it's been tapped. In practice that would mean that data retention would be of the same use that the present tapping legislation, so it presents only a cost with no benefit.

If the company is outside the EU, not even tapping can be done. Google created a public use VPN with their Google Wifi project. While their client software isn't provided by Google anymore, it can still be downloaded in sites like Softpedia. In this situation, any EU citizen can simply download their software and use the Internet without worrying about data retention in the EU.

Anonymizing proxy services: Web

There are anonymizer proxies that provide anonymous web access, using standard e-commerce cryptography to communicate with the final user and cloaking the Internet address of the user. As in the case above, the data retained is only the connection to the server.

A list of such proxies can be found in


Most hosting companies (Pair Networks, Inc. in the US, for instance), provide a secure interface to their webmail system. This means that any EU citizen that pays for their services can read and send email without any information about it being recorded, other than a secure (e-commerce like) connection to the hosting company.

P2P communications

Some P2P services, being it file transfer or voice over IP, use other computers to allow communication between computers behind firewalls. This means that trying to follow a call between two citizens might, mistakenly, identify a third citizen unaware of the communication.

Privacy enhancing tools

For security conscious citizens, using tools like Tor, mixmaster and the cryptography options integrated into any major mail client these days is always an option. Though, in practice, not everyone has the knowledge to employ the techniques described above.

Tor is a project of the USA's Electronic Frontier Foundation to create an onion routing based P2P network where normal protocols are made untraceable. Mixmaster is a remailer service that allows anonymous email sending.

JAP is a project very similar to Tor. It is designed to route web requests through several proxies to hide the end users Internet address. Tor support has been included into JAP.

Making use of Dynamic IP Addresses

Almost since the beginning of commercial Internet dynamic IP addresses were given out to consumers out of a pool of a set of addresses. Every connection is logged to a central database. When broadband DSL made PPPoE popular one would still be given dynamic IP addresses but disconnecting the session and calling back was now a matter of seconds instead of waiting a minute for an analog modem to retrain. If a large number of computers disconnected and reconnected every minute this would cause a lot of logs to be retained until it becomes uneconomic to retain this data of logins. Web usually still works with this and so does email. In calculation: if 1 million users at an ISP connected and disconnected every minute and did this 24/7 an ISP would need 11 terabytes of storage, for 365 days of retention, if they sign a 4 byte field for logon/logoff time, a 4 byte field for customer number and a 4 byte field for IP address used.

See also

External links

This Text is based on the article Telecommunications data retention from the free encyclopedia Wikipedia and has been released under the GNU Free Documentation License (not a Creative Commons-License). In Wikipedia a list of authors is available.