2015-01-22

Mendacity from Amy Nicholson

In Slate, L.A. Weekly movie critic Amy Nicholson takes aim at deceased sniper and Navy SEAL Chris Kyle:

Take American Sniper, one of the most mendacious movies of 2014. Clint Eastwood was caught in a trap: His subject, murdered Navy SEAL Chris Kyle, lied a lot. In his autobiography, he said he killed two carjackers in Texas, sniped looters during Hurricane Katrina, and punched Jesse Ventura in the face. None of that was true. So Eastwood was stuck. Should he repeat Kyle’s lies as truth? Expose him as a liar?
Ironically her article is titled "Clint Eastwood's American Sniper is one of the most mendacious movies of 2014", because she clearly hasn't read Kyle's autobiography. In his autobiography he does not discuss either of the first two situations she describes, at all. The third situation is described, but Jesse Ventura is not mentioned (Kyle calls the participant "Scruffy" and although some of Scruffy's background is consistent with Ventura's, it's not an obvious link). So Nicholson seems happy with at least one of two situations: 1. making claims about a book she hasn't read, or 2) making knowingly false claims about a book she has read.

It's slightly clearer when you read the New Yorker article which she links because they report third person recounting of the first two stories: people who claim to have heard Kyle talk about them. Kyle may or may not have told these stories, and they may or may not have been accurately recounted by the third parties. The Scruffy story was later confirmed by Kyle in a video interview to pertain to Ventura, and a court subsequently decided that Ventura had been libelled by it. It's a pretty misleading recounting by Nicholson though, whether or not the claim turns out to be substantially true - if you aspire to being an actual journalist, one would expect you to have a clear understanding of 1st vs 2nd vs 3rd party sources and make the distinction clear in your articles. Perhaps Ms. Nicolson has no such aspiration and is happy being a partisan hack.

2015-01-06

BBC booze bill shocker

The shocker is, it's extremely reasonable:

The Corporation stated that the figure related to 'non-production related and production related spend'.
It added: 'The total spent on alcohol for the period 1st October 2013 to 26th October 2014 with the BBC's single preferred supplier Majestic Wine PLC was £43,000.'

I'm not the greatest fan of the BBC's compulsory TV licence, but I really don't think that this is worthy even of a Daily Mail throwaway article:

  • Use of bulk supplier for savings: check
  • Cost per employee per year: £2 , eminently reasonable, no reason to think this is taxpayer-funded employee booze
  • Cost per day: £130 over all channels and events. That's about 3 bottles of Veuve Clicquot NV at Sainsbury's prices. Assuming the BBC allocates half a bottle per top echelon (MP, MEP, sleb) guest, that's 6 top echelon guests per day which sounds about right.
It comes as up to 50 MPs called for the licence fee to be scrapped and replaced with a voluntary subscription service in its place.
Talk about tenuous connections. This is possibly one of the strongest signals of thrifty BBC spending there is, and you're linking it to a call for licence fee repeal? Your logic is not like our Earth logic, Daily Mail.

2014-12-24

Scentrics, "Key Man" and mobile security, oh my

From a story in the Daily Mail today I found this October article in the Evening Standard about security firm Scentrics which has been working with UCL

In technical parlance, Scentrics has patented the IP for “a standards-based, fully automatic, cryptographic key management and distribution protocol for UMTS and TCP/IP”. What that translates as in layman’s language is “one-click privacy”, the pressing of a button to guarantee absolute security.
Where issues of national security are concerned, the ciphers used are all government-approved, which means messages can be accessed if they need to be by the security services. What it also signals in reality is a fortune for Scentrics and its wealthy individual shareholders, who each put in £500,000 to £10 million.
Hmm. That's a fairly vague description - the "government-approved" language makes it look like key escrow, but it's not clear. I was curious about the details, but there didn't seem to be any linked from the stories. Chandrasekaran was also touting this in the Independent in October, and it's not clear why the Mail ran with the story now.

I tried googling around for any previous news from Scentrics. Nada. So I tried "Paran Chandrasekaran" and found him back in 2000 talking about maybe netting £450M from the prospective sale of his company Indicii Salus. I couldn't find any announcements about the sale happening, but it looks like email security firm Comodo acquired the IP from Indicii Salus in March 2006. According to Comodo's press release

The core technology acquired under this acquisition includes Indicii Salus Limited's flagship security solution which, unlike other PKI offerings, is based on server-centric architecture with all information held securely in a central location thus providing a central platform necessary to host and administer central key management solutions.
That's a single-point-of-failure design of course - when your central server is down, you are screwed, and all clients need to be able to authenticate your central server so they all need its current public key or similar signature validation. It's not really world-setting-on-fire, but hey it's 8 years ago.

Then LexisWeb turns up an interesting court case: Indicii Salus Ltd v Chandrasekaran and others with summary "Claimant [Indicii Salus] alleging defendants [Chandrasekaran and others] intending to improperly use its software - Search order being executed against defendants - Defendants applying to discharge order - Action being disposed of by undertakings not to improperly use software"

Where the claimant had brought proceedings against the defendants, alleging that they intended to improperly use its software in a new business, the defendants' application to discharge a search order, permitting a search of the matrimonial home of the first and second defendants, would be dismissed.
The case appears to be fairly widely quoted in discussions of search+seizure litigation. I wonder whether Paran Chandrasekaran was one of the defendants here, or whether they were other family members? There's no indications of what happened subsequently.

How odd. Anyway, here's a sample of the Scentrics patent (USPTO Patent Application 20140082348):

The invention extends to a mobile device configured to:
send to a messaging server, or receive from a messaging server, an electronic message which is encrypted with a messaging key;
encrypt a copy of the message with a monitoring key different from the messaging key; and
send the encrypted copy to a monitoring server remote from the messaging server.
[...]
Thus it will be seen by those skilled in the art that, in accordance with the invention, an encrypted copy of a message sent securely from the mobile device, or received securely by it, is generated by the device itself, and is sent to a monitoring server, where it can be decrypted by an authorized third party who has access to a decryption key associated with the monitoring key. In this way, an authorized third party can, when needed, monitor a message without the operator of the messaging server being required to participate in the monitoring process.
Because both the message and its copy are encrypted when in transit to or from the mobile device, unauthorized eavesdropping by malicious parties is still prevented.
This reads to me like "given a message and a target, you encrypt it with a public key whose private key is held by your target and send it to the target as normal, but you also encrypt it with a separate key known to a potential authorized snooper and send it to their server so that they can access if they want to."

WTF? That's really not a world-beating million-dollar idea. Really, really it's not. Am I reading the wrong patent here? Speaking personally, I wouldn't invest in this idea with five quid I found on the street.

2014-12-16

The 2038 problem

I was inspired - perhaps that's not quite the right word - by this article on the Year 2038 bug in the Daily Mail:

Will computers be wiped out on 19 January 2038? Outdated PC systems will not be able to cope with time and date, experts warn Psy's Gangnam Style was recently viewed so many times on YouTube that the site had to upgrade the way figures are shown on the site.
  1. The site 'broke' because it runs on a 32-bit system, which uses four-bytes
  2. These systems can only handle a finite number of binary digits
  3. A four-byte format assumes time began on 1 January, 1970, at 12:00:00
  4. At 03:14:07 UTC on Tuesday, 19 January 2038, the maximum number of seconds that a 32-bit system can handle will have passed since this date
  5. This will cause computers to run negative numbers, and dates [sic]
  6. Anomaly could cause software to crash and computers to be wiped out
I've numbered the points for ease of reference. Let's explain to author Victoria Woollaston (Deputy Science and Technology editor) where she went wrong. The starting axiom is that you can represent 4,294,967,296 distinct numbers with 32 binary digits of information.

1. YouTube didn't (as far as I can see) "break".

Here's the original YouTube post on the event on Dec 1st:

We never thought a video would be watched in numbers greater than a 32-bit integer (=2,147,483,647 views), but that was before we met PSY. "Gangnam Style" has been viewed so many times we had to upgrade to a 64-bit integer (9,223,372,036,854,775,808)!
When they say "integer" they mean it in the correct mathematical sense: a whole number which may be negative, 0 or positive. Although 32 bits can represent 4bn+ numbers as noted above, if you need to represent negative numbers as well as positive then you need to reserve one of those bits to represent that information (all readers about to comment about two's complement representation can save themselves the effort, the difference isn't material.) That leaves you just over 2bn positive and 2bn negative numbers. It's a little bit surprising that they chose to use integers rather than unsigned (natural) numbers as negative view counts don't make sense but hey, whatever.
Presumably they saw Gangnam Style reach 2 billion views and decided to pre-emptively upgrade their views field from signed 32 bit to signed 64 bit. This is likely not a trivial change - if you're using a regular database, you'd do it via a schema change that requires reprocessing the entire database, and I'd guess that YouTube's database is quite big but it seemed to be in place by the time we hit the signed 32 bit integer limit.

2. All systems can only handle a finite number of binary digits.

For fuck's sake. We don't have infinite storage anywhere in the world. The problem is that the finite number of binary digits (32) in 4-byte representation is too small. 8 byte representation has twice the number of binary digits (64, which is still finite) and so can represent many more numbers.

3. The number of bytes has no relationship to the information it represents.

Unix computers (Linux, BSD, OS X etc.) represent time as seconds since the epoch. The epoch is defined as 00:00:00 Coordinated Universal Time (UTC - for most purposes, the same as GMT), Thursday, 1 January 1970. The Unix standard was to count those seconds in a 32 bit signed integer. Now it's clear that 03:14:08 UTC on 19 January 2038 will see that number of seconds exceed what can be stored in a 32 bit signed integer, and the counter will wrap around to a negative number. What happens then is anyone's guess and very application dependent, but it's probably not good.
There is a move towards 64-bit computing in the Unix world, which will include migration of these time representations to 64 bit. Because this move is happening now, we have 23 years to complete it before we reach our Armageddon date. I don't expect there to be many 32 bit systems left operating by then - their memory will be rotted, their disk drives stuck. Only emulated systems will be still working, and everyone knows about the 2038 problem.

4. Basically correct, if grammatically poor

5. Who taught you English, headline writer?

As noted above, what will actually happen on the date in question is heavily dependent on how each program using the information behaves. The most likely result is a crash of some form, but you might see corruption of data before that happens. It won't be good. Luckily it's easy to test programs by just advancing the clock forwards and seeing what happens when the time ticks over. Don't try this on a live system, however.

6. Software crash, sure. Computer being "wiped out"? Unlikely

I can see certain circumstances where a negative date could cause a hard drive to be wiped, but I'd expect it to be more common for hard drives to be filled up - if a janitor process is cleaning up old files, it'll look for files with modification time below a certain value (say, all files older than 5 minutes ago). Files created before the positive-to-negative date point won't be cleaned up by janitors running after that point. So we leave those stale files lying around, but files created after that will still be eligible for clean-up - they have a negative time which is less than the janitor's negative measurement point.

I'm sure there will be date-related breakage as we approach 2038 - if a bank system managers 10 year bonds, then we will see breakage as their expiry time goes past january 2038, so the bank will see breakage in 2028. But hey, companies are already selling 50 year bonds so bank systems have had to deal with this problem already.

Thank goodness that I can rely on the Daily Mail journalists' expertise in all the articles that I don't actually know anything about.

2014-12-08

2014-12-05

Whoda thunk? An actual piece of journalism on the University of Virginia "frat house gang rape" story

It seems as if the wheels are coming off Sabrina Rubin Erdely's story in Rolling Stone of gang rape on the University of Virginia's campus.

In the face of new information, there now appear to be discrepancies in Jackie's account, and we have come to the conclusion that our trust in her [my italics] was misplaced. [...] We are taking this seriously and apologize to anyone who was affected by the story.
That's certainly a novel way of writing "our unquestioning acceptance of her decidedly dodgy tale" and "had their reputations dragged through the dirt in the national media".

My favourite wonk, Megan McArdle, has a must-read piece on how this happened and how the crazy rush to publish a decidedly dodgy and unverified story has been one of the worst things to happen to real campus rape victims in a long time:

So now the next time a rape victim tells her story to a journalist, they will both be trying to reach an audience that remembers the problems with this article, and the Duke lacrosse case, and wonders if any of these stories are ever true. That inference will be grotesquely false, but it is the predictable result of accepting sensational stories without carefully checking. The greatest damage this article has done is not to journalism, or even to Rolling Stone. It is to the righteous fight for rape victims everywhere.
Go read the whole thing, and despair at the media environment that splashed Erdely's story over the national news but will fail to discuss the points in McArdle's article in anything but the most oblique terms.

2014-11-26

Unexpected consequences of Obamacare and immigration amnesties

I'm not sure why this hasn't generated more outrage yet: the Washington Times has spotted that President Obama's plan to legalize employment for illegal immigrants might screw over American workers even more than initially suspected:

President Obama's temporary amnesty, which lasts three years, declares up to 5 million illegal immigrants to be lawfully in the country and eligible for work permits, but it still deems them ineligible for public benefits such as buying insurance on Obamacare's health exchanges.
Seems sensible enough, although the amnesty beneficiaries might well be eligible for the Earned Income Tax Credit if they have kids. But there's a consequence for the lack of health exchange rights:
Under the Affordable Care Act, that means businesses who hire them won't have to pay a penalty for not providing them health coverage [my emphasis] — making them $3,000 more attractive than a similar native-born worker, whom the business by law would have to cover.
Oopsie. Since the immigrants will tend to participate in the lower-paid end of the employment spectrum, that means the $3000 delta will be a huge fraction of the wage. That's quite the competitive advantage. Sure, it means in practice that they won't have ACA-compliant health care - and in fact I'd expect many employers to pay their amnestied workers a higher headline wage to compensate for this lack of employer-supported healthcare. Nevertheless, once it's legal to employ these workers openly, the wage differential makes them look very attractive.

This won't affect unionized jobs where wages can't easily be varied, but in the private sector the medium-sized businesses who have more than 50 employees will start sucking up all the amnestied labor they can and will stop hiring the locals. Small businesses which have pushed workers into part-time slots to avoid the ACA can now replace two part-time workers with a full-time amnestied worker.

This is what happens when you create a baroque, complicated legal framework for employment and health insurance. When you subsequently make changes, you will find that they have unexpected effects.

2014-11-23

Anatomy of a timeshare sale

Dear readers, the things I do on your behalf. Herewith my notes from participating in a recent timeshare sales session which was the condition of a fairly well discounted holiday which my partner and I recently enjoyed.

The vacation property itself was very pretty - manicured lawns, artfully trimmed flowering bushes and a background of blue skies and the sound of crashing waves. The sales office itself was tucked away in a corner of the imposing main clubhouse, presumably because once you’re an owner you don’t like to be reminded of how and where they got you. It was a reasonably high traffic operation, several other couples there waiting or coming through - note that there were no singles, only couples. I'd guess they’re maximising their chances of finding a weak spot and then leveraging it to pressure the other party. Divide and conquer FTW!

The waiting room had the usual free beverages to enjoy for the few minutes we were waiting. Coffee was from a press-top urn and was awful. Normally I'm OK with urn coffee in a pinch, but my goodness this stuff was dreadful; I had to fall back to Lipton tea. This was scheduled to be a 2 hour session so my tolerance for coffee absence would be tested to its limit.

I'll call our sales rep "Nick", who was audibly a New Yorker. He led us down to his office and the presentation started after a few minutes of soft soap "how was your vacation so far? what have you enjoyed?" which was fairly obviously an intelligence-gathering exercise.

Nick started the sell emphasising that this was not a high pressure sales session. He then described the "price integrity" of his company, that they never discounted or negotiated on price (yeah, sure, you betcha snookums) and referenced back to how much we'd enjoyed the holiday so far to stimulate the guilt gland. He then noted the extra financial incentives if we bought right now, today, with a yes/no decision at the end of the session. What was that about "no high pressure sales", Nick? He outlined our aim today which was to decide whether our future vacations would be better with or without TIMESHARECO ownership, which was studiously neutral so far. At the end of the session we would be meeting with the company inventory manager for details on prices, incentives etc.

About 10-15 minutes in and Nick took a break to "get some water". Presumably this was to check with his boss on his boss's read on the situation so far. I didn't think to check for a video or audio monitor in the office; nothing was obvious, and I'm guessing that there wasn't any eavesdropping going on. Certainly nothing subsequently made me suspect that.

Nick started the next session reviewing our past holidays and latched on to our holiday last year as similar to the kind of thing he was selling. He asked us to name our "dream" 3-5 money-no-object vacations which we did. He picked out quality as a factor in our holidays and started talking numbers on room prices, picking a $200/night base price.

We learned after casual conversation from me that he had retired from a job as a retirement plan sales manager, but had come back into the timeshare sales game after a couple of years. In light of the later discussions, this made a lot of sense. He likened the scheme he was selling as a "401(k)" (money purchase pension scheme) for holidays - invest money and get a steady yield of vacations.

During the meeting he took very short but effective notes on a single sheet of paper, only a few words per concept; around now he read back to us a summary of what he'd noted, and pretty much nailed everything. I was very impressed at his technical skill. I also approved of the strategic placing of his office with a genuinely lovely garden and waterfall view - he sat with his back to it, so it clearly wasn't intended for his benefit. I bet the room views aren't like that (except for the show rooms.)

Now we come on to the numbers. He was trying to sell on the basis of 7 days stay, $200/night, over 20 years - that if we did this with his company then it would be cheaper than renting a hotel room each year. He presented a table showing cost of hotel rooms in brackets - but quoting in non-constant dollars. The chart spanned 40 years - so starting from the mid-1970 when 11% annual inflation was the average - but actually only 7% over past 10 years (I did the math). Later, checking the US inflation calculator it's clear that 1974-1984 is by far the steepest inflation decade of the past 40 year - 110% compared to 42% (1984-1994), 27% (1994-2004) or 25% (2004-2014).

I innocently asked him "but aren't wages inflating too, so shouldn't this be expressed in constant dollars or at least expressed in terms of purchasing power? And aren't hotel prices determined by supply/demand - what you can persuade people to pay, not what your costs are, so heavily influenced by wages?" at which point he pretended confusion. I also asked why he was looking at a 40 year basis when we were talking about a future span of 20 years, which met with a similar response.

Now it makes sense why he used to sell retirement plans... he's essentially selling a financial plan. He's saying that if we give TIMESHARECO about 20 grand then they can invest it in property and meet the cost of our stays over the next 20 years while presumably turning a small profit including his commission. And yet, they can't persuade the major financial establishments to make the same investments and profit directly. I wonder why?

Now the "here's all the places you can stay!" list. About 70 locations in 10 countries - not a massive amount, but they have "affiliates" in 100 countries with over 5000 resorts you can stay at. Minimum of 3 nights per stay, no max, which seems reasonable. With your purchase of the plan you get X points per year to spend on properties, and can transfer points between years. It costs $100 to carry forward non-spent points, but $0 to borrow them from future years - cheaper to take a loan than save up. What's wrong with this picture? It means that they want the additional money they get from you actually staying, of which more later.

We toured through photo sets of properties in countries we might visit, though only TIMESHARECO properties not affiliates - which was a nice sleight of hand. Apparently TIMESHARECO "reviews" the quality of the affiliates to ensure they're up to scratch. I'm sure you're as reassured as I was. It's a first-come first-served model for all properties. Nick claimed that there was a low probability of all affiliate properties being full in an area even in busy time e.g. spring break but didn't address TIMESHARECO numbers directly. So they almost certainly have a problem with availability during this times. Affiliates charges $200 per booking which is a nice little earner and pushes you towards fewer, longer holidays in affiliates.

He gave us a brochure for the affiliate program: RCI. According to their SSL cert information they are Wyndham Worldwide Corporation based in Parsippany, New Jersey. Their stock is up about 15% y/y so clearly the timeshare business is doing well out of the boom.

Nick took another break, this time more extended than the previous one, presumably to allow replanning of his sales approach. I couldn't help but notice that he didn't offer us a refill of our beverages.

He mentioned in passing that there was also a maintenance fee which covers insurance for the property, in response to an earlier question I had about "what if the property we buy rights to burns down?" We fenced for a few minutes, then 70 mins after the start of the discussion he gave up, said that we didn't have to tour the property if we didn't want to - we didn't - and handed over the bonus gifts that we were due to receive at the end of the property. He did try a last gasp attempt with vacation offer similar to what we had already enjoyed, with another timeshare presentation linked in. I'm sure that if we'd taken this up then we'd have been lined up with their Top Gun negotiator. But we said no thanks, and left.

Overall a fascinating view into the world of timeshare sales. I didn't feel in any danger of buying at any point, but I give Nick his due that he tried very hard and used most of the tricks in the book without resorting to what I'd regard as "high pressure" sales. Perhaps the fact that I was taking notes alarmed him a little; he emphasised at the start that he'd give us all the items discussed in writing, but of course with us leaving before closing this didn't happen (if it would have happened). Credit to him that he recognised when he was beaten and didn't waste our time or his beyond that point. It also turned out to be remarkably easy to elicit information about him and divert him off course for a few minutes. Presumably this was because he thought that he was making a social connection and common ground.

The offer itself of course was completely overpriced - I checked out the secondary market in TIMESHARECO properties and they were a) heavily discounted, around 60% of face value and b) not selling, though of course these are related and just give you a ballpark idea of the market clearing price. The annual maintenance cost was around $1300 - i.e. the same as 6 nights of hotel stays at $200/night. If you buy in the primary market, you are a total mug or you have lots of money, the holiday model fits you and you don't mind paying a healthy excess for the convenience.

2014-11-12

Lipstick on a postal pig

I can't help but share this lunacy with you. The (American) Center For Economic and Policy Research thinks that the problem with the US Postal Service isn't the lackadaisical, contemptuous, inefficient distribution of mail which it perpetrates. It's just not properly utilized. Instead, we should allow it to run banking services at the same efficiency with which it delivers mail:

[...] the Postal Service could improve its finances by expanding rather than contracting. Specifically, it can return to providing basic banking services, as it did in the past and many other postal systems still do. This course has been suggested by the Postal Service's Inspector General.
This route takes advantage of the fact that the Postal Service has buildings in nearly every neighborhood in the country. These offices can be used to provide basic services to a large unbanked population that often can't afford fees associated with low balance accounts. As a result they often end up paying exorbitant fees to check cashing services, pay day lenders and other non-bank providers of financial services.
Of course, the reason that banks have run a mile from providing banking services to clients with low income or dubious immigration status, running away from a steady (albeit low) income stream, is due to... government regulatory pressure. Who'd have thought that the government would have caused these problems?

Now the CEPR is proposing that a government agency can step in and fix the very real problems in banking access that other government agencies have created. I don't know whether to laugh or cry.

Incidentally, my personal experience with sending mail through the USPS - a monthly mail to a residential address within the same state, dropped in a regular post box - is that the failure rate is about 1 in 13. This is corroborated by the experience of The Advice Goddess (Los Angeles resident Amy Alkon, if you're not reading her blog or buying her books then you really should):

There is no way that the USPS could comply with the existing banking regulations in the USA without having the same order of overhead as the major US banks. I suspect their savings in property costs are insignificant; even if they could train existing post office counter staff to be bank tellers as well without any major salary inflation, all the backend systems and personnel required would kill their cost advantage. Check out the USPS compensation and benefits: "regular salary increases" means you're paid by length of service, not productivity, they get federal health benefits which are a step or three above Obamacare coverage, and they get a defined benefit retirement plan. Believe me, if you're staff at a major bank, you would sell your mother on the streets to get these benefits.

All the CEPR is doing in this article is lobbying for an increase in (unionized) federal government employees. The government, and therefore the taxpayer, is going to pick up the tab, but that's Just Fine with them. The only way I can see this working is if the USPS is exempted from most of the existing banking regulations - and if that's the problem, why not just repeal them for everyone else as well?

2014-11-04

A caricature of Civil Service placement and rhetoric

The new director of GCHQ was announced earlier this year as Robert Hannigan, CMG (Cross of St Michael and St George, aka "Call Me God") replacing the incumbent Sir Iain Lobban, KCMG (Knight's Cross of St Michael and St George, aka "Kindly Call Me God"). Whereas Sir Iain was a 30 year veteran of GCHQ, working his way up from a language specialist post, Hannigan was an Oxford classicist - ironically at Wadham, one of the few socialist bastions of the university - and worked his way around various government communications and political director posts before landing a security/intelligence billet at the Cabinet office. Hannigan is almost a cliché of the professional civil servant.

Hannigan decided to write in the FT about why Facebook, Twitter and Google increasing user security was a Bad Thing:

The extremists of Isis use messaging and social media services such as Twitter, Facebook and WhatsApp, and a language their peers understand. The videos they post of themselves attacking towns, firing weapons or detonating explosives have a self-conscious online gaming quality. [...] There is no need for today’s would-be jihadis to seek out restricted websites with secret passwords: they can follow other young people posting their adventures in Syria as they would anywhere else.
Right - but the UK or US governments can already submit requests to gain access to specific information stored by Facebook, Google, Twitter et al. What Hannigan leaves out is: why is this not sufficient? The answer, of course, is that it's hard to know where to look. Far easier to cast a dragnet through Internet traffic, identify likely sources of extremism, and use intelligence based on their details to ask for specific data from Facebook, Google, Twitter et al. But for the UK in the first half of 2014, the UK issued over 2000 individual requests for data, covering an average of 1.3 people per request. How many terrorism-related arrests (never mind convictions) correspond to this - single digits? That's a pretty broad net for a very small number of actual offenders.

Hannigan subsequently received a bitchslap in Comment is Free from Libdem Julian Huppert:

Take the invention of the radio or the telephone. These transformed the nature of communication, allowing people to speak with one another across long distances far more quickly than could have ever been imagined. However, they also meant that those wishing to do us harm, whether petty criminals or terrorists, could communicate with each other much more quickly too. But you wouldn’t blame radio or phone manufacturers for allowing criminals to speak to each other any more than you would old Royal Mail responsible for a letter being posted from one criminal to another.
Good Lord, I'm agreeing with a Libdem MP writing in CiF. I need to have a lie down.

Hannigan is so dangerous in his new role because he's never really had to be accountable to voters (since he's not a politician), nor influenced by the experience and caution of the senior technical staff in GCHQ (since he never worked there). He can view GCHQ as a factory for producing intelligence to be consumed by the civil service, not as a dangerous-but-necessary-in-limited-circumstances intrusion into the private lives of UK citizens. After all, he knows that no-one is going to tap his phone or read his email.

Personally, I'd like to see a set of 10 MPs, selected by public lottery (much like the National Lottery draw, to enforce fairness) read in on GCHQ and similar agency information requests. They'd get to see a monthly summary of the requests made and information produced, and would be obliged to give an annual public report (restricted to generalities, and maybe conducted 6 months in arrears of the requests to give time for data to firm up) on their perception of the width of the requests vs information retrieved. That's about 40 Facebook personal data trawls per MP, which is a reasonably broad view of data without excessive work. Incidentally, I'd also be interested in a breakdown of the immigration status of the people under surveillance.