It's time to look at how UKIP have done during the three months while I've been away. But it's hardly as if I can claim any great insight here. I didn't see the durability of the UKIP surge and my expectation when I first wrote about UKIP in April was that they were likely to fade. At that time, I wrote:
"It seems that the bookies are catering to the enthusiasm of the purple punters by relieving them of their money at poor odds."
At that time, only five seats had UKIP at odds of less than 10/1. As of today's date, there are 28 such seats (and neither Clacton nor Rochester & Strood, which must both be among UKIP's best bets, have general election markets as of today). So that shows how good a judge I was of how the wind is blowing.
I'll have to chalk that up to experience and I'll have to show some humility (there's a first time for everything, of course). So let's leave the inferences aside and instead let's look at the current state of play. Here are UKIP's prospects ranked in order of their present odds:
If we compare this to the position merely six months ago, UKIP have been the punters' darlings in the interim:
And here is the position in June - UKIP have generally shortened even since then (when 19 seats were at a price of less than 10/1):
I shall first try to look at where UKIP has been shortening. Because the shortening prices have not been uniform. UKIP are at the same price in Eastleigh as they were six months ago. They're the same price in Castle Point and Louth & Horncastle as they were three months ago. They've actually drifted out in Folkestone & Hythe in the last three months.
So what's been happening in the markets?
1) Punters think that UKIP is developing local strongholds capable of being converted into seats.
Five Kent seats are priced at under 10/1. There is no general election market for Rochester & Strood at present, and odds on that would certainly be below 10/1, whether or not UKIP ultimately wins its by-election. Three more seats in the county are priced at under 16/1. That's nine out of the 17 constituencies in the county - every coastal constituency in the county other than Canterbury and Faversham & Mid Kent.
Essex is a similar story. Three constituencies are under 10/1 (not including Clacton, which looks like UKIP's best bet at the next election, but for which there is no market). A further eight constituencies are priced at 16/1 or under. So 12 out of 18 constituencies in Essex are seen as at least outside chances for the Kippers, including every coastal constituency except Witham.
Indeed, over the county boundary in London but still along the Thames estuary, both Dagenham & Rainham and Bexleyheath & Crayford also feature on the latest list of shorter priced UKIP targets. The Thames estuary is hotting up for the purple team.
Incidentally, for those believing that UKIP is attracting None Of The Above voters from the Lib Dems, it's worth noting that the Thames estuary was one of the Lib Dems' weakest areas at the last election. There isn't a single seat abutting the water from Bethnal Green & Bow to Castle Point on the north side inclusive and from Erith & Thamesmead to Gravesham on the south side inclusive where the Lib Dems tallie more than 15%. There's more than that going on.
But it's not the only hotspot for UKIP. Four seats in the old borders of Lincolnshire are priced at under 10/1, and a further three seats at 16/1. That's all four of the Humberside constituencies south of the Humber and all three coastal constituencies in the new borders of Lincolnshire. And indeed, most of the seats of what I have previously called the Saxon Shore feature in the list:
2) Punters are giving some love to the idea of UKIP making progress in Labour urban strongholds
Most of the shortest priced seats are Conservative held, but there are five Labour-held seats where UKIP are priced at under 10/1. A further 21 Labour-held seats are priced between 10/1 and 16/1.
I must confess that I have found this price movement in particular difficult to follow, since no such constituency has had an opinion poll showing UKIP likely to take such a seat. Even Rotherham, a seat whose recent history makes it as likely to consider an insurgent party as any such seat, has shown a double digit lead for Labour over UKIP in a recent Survation constituency poll. This does not suggest to me that a general breakthrough in Labour-held seats is particularly plausible without a powerful new boost to UKIP's support among Labour's traditional voter base.
That said, the Heywood & Middleton by-election result does suggest a rethink may be necessary. UKIP will need to think about how to harness the anger that they have undoubtedly found in such constituencies.
3) Punters are following the Survation and Lord Ashcroft constituency polls with interest
Alan Bown, on behalf of UKIP, has commissioned a series of opinion polls in individual constituencies. 11 have been made public (it is rumoured that several more have been commissioned in the seats of possible Conservative defectors, though I have no direct knowledge either way of this). Additionally, Survation surveyed both Clacton and Rochester & Strood for newspapers.
These constituencies are Great Grimsby, Dudley North, South Thanet, Folkestone & Hythe, Bognor Regis & Littlehampton, Great Yarmouth, Crewe & Nantwich, Eastleigh, North Thanet, Rotherham, Boston & Skegness, Rochester & Strood and Clacton. With the exception of Crewe & Nantwich, which had been included for control purposes anyway, all of these seats have UKIP at 8/1 or less.
There is of course an element of circularity - Alan Bown had these seats polled because they were seen as serious prospects.
Separately, Lord Ashcroft has also been polling various constituencies. He has not been focusing on UKIP particularly, but has looked at Great Yarmouth, South Thanet, Eastleigh (twice), Thurrock, Plymouth Sutton & Devonport, St Austell & Newquay (twice) and Camborne & Redruth. Lord Ashcroft found leads for UKIP in South Thanet and Thurrock, but in neither of these cases were the leads substantial. In Great Yarmouth, unlike Survation, Lord Ashcroft found that UKIP were a close second behind the Conservatives (Survation found the order to be UKIP, Labour, Conservative). In Eastleigh, Lord Ashcroft (unlike Survation) found UKIP were a fairly distant third behind the Lib Dems and the Conservatives. In the Cornish seats, Lord Ashcroft found that UKIP were second behind the Conservatives. UKIP were a feature in Plymouth Sutton & Devonport without yet seriously challenging for the seat.
These findings have all percolated into the pricing of the individual seats.
And what's happening in reality?
If I knew that, I'd be a happy man. We do not want for clues, but what is much harder is interpreting those clues.
Decoding the constituency polls
Let's start with the polling. I find this really hard to interpret. For starters, I have big reservations about constituency polls as a general concept. It's one thing trying to get a balanced sample across the country, where pollsters have a lot of practice. It's another thing trying to do so in a specific constituency where the appropriate balanced sample will differ quite significantly from that across the nation as a whole. This problem is amplified where a new party is on the scene, with new demographic indicators to need to balance.
Then there are the specific pollsters. Let me be clear: I regard both Lord Ashcroft and Survation as reputable pollsters, honestly trying to get the most accurate results that they can. Both do great work. But we would expect different results from different polling companies with their different methodologies. We need to understand that when interpreting these polls.
Survation in particular have consistently reported the highest levels of support for UKIP of any pollster when they conduct national opinion polls. Lord Ashcroft's national polls also report UKIP at the higher end of the scale: his last five national polls record UKIP support at 18%, 14%, 17%, 17%, 17% (an average of 16.6%). The UK Polling Report average at the date of writing is 15%. Note, this does not suggest that either Survation or Lord Ashcroft are necessarily getting the wrong result. But when judging UKIP's prospects in individual constituencies off the back of these constituency polls, we need to be aware that we are seeing the results presented in a light that is likely to represent the most hopeful view for UKIP.
Can we test Survation's and Lord Ashcroft's polling results? Well, to a limited extent we can. Survation's polls for the Euro elections overstated UKIP's tally by 5%.
Both Survation and Lord Ashcroft produced constituency polls for the Newark by-election as well. Both overstated both UKIP and Labour by a bit, and understated the Conservatives - in the case of Survation, by a lot. Of course, there may have been further movements in voting intentions after the polls were taken, but that's easier to understand in terms of the Conservatives being understated and Labour being overstated if the Conservatives (and indeed UKIP) were squeezing Labour, and UKIP being overstated is a little harder to understand.
Both also produced polls for both the Clacton and the Heywood & Middleton by-elections. These fared much better in Clacton (again Labour were a bit overstated by both, but both pollsters were more or less right about the Conservatives). Lord Ashcroft underestimated UKIP a bit, Survation overestimated UKIP a bit.
Heywood & Middleton was a very different story. UKIP did far better than either pollster had envisaged and the Conservatives did worse. In fairness, Survation stress in their poll that it's a nowcast not a forecast, and I expect Lord Ashcroft would say the same. Given that Nigel Farage reportedly stayed away from the constituency on the day because he thought that UKIP had no chance of taking the seat, it seems likely that there was a late swing to them that not even UKIP were expecting. Again, Labour's poll share was significantly overestimated. Perhaps it shows the difficulty of constituency polling and the dangers of relying on it too closely.
Of course any opinion poll is just a snapshot of the current position. Even where UKIP are in the lead, they may be overtaken by another party successfully securing tactical votes to keep them out. Or UKIP may, as in Heywood & Middleton, surge.
With all these notes of caution, these polls are still an invaluable resource. And I am drawn to two bets on UKIP off the back of them. If UKIP are 20 points ahead in Boston & Skegness, even with a pollster that discloses the most hopeful results for them, 10/11 on them is a very solid bet (I note that the odds have now shortened to 4/5, but it remains a good bet).
One more note of caution in this particular constituency: the sample in the poll looks odd - on the face of it, barely a quarter of those polled voted for one of the Conservatives, Labour, the Lib Dems or UKIP in 2010. But there's a lot of leeway for peculiarities in a 20 point lead. I'm aware that isam of political betting likes this bet for the same reason, and he drew it to my attention, so I need to give full credit to him.
And 13/8 on UKIP in Thurrock looks like a good bet, given that Lord Ashcroft found that they were in the lead at present. That may or may not be correct now, and they may or may not be overtaken. But 13/8 on what on our best information seems like a front runner will do me.
Conversely, the 4/6 on Nigel Farage in South Thanet looks overdone. With the Conservatives and Labour both apparently in hot pursuit if Lord Ashcroft's polling is to be believed, the risk that opposition to him personally might coalesce around one or other of these as a pre-capitation strategy looks substantial.
What's happening on the ground?
Again, this is very hard to know from the outside. The kippers seem chipper, and on those political blogs where they hang out, startling predictions are made of hoped-for gains that seem hard to credit. But then, a 20 point lead for UKIP in Boston & Skegness is also startling.
From the by-elections, we can conclude the following:
1) The Conservatives' attacks on UKIP in Clacton were of no use whatsoever. Be under no illusions: the Conservatives made a fairly significant effort, including a Prime Ministerial visit. They had hoped to close the majority so that they stood a chance of taking it in the general election (I heard from two separate sources on the day of the by-election that they hoped to keep the majority below 5,000). They did not, and indeed they seem to have had a poor handle of what was happening on the ground in reality.
2) UKIP don't know their own strength. The Telegraph reported today that "Mr Farage chose not to campaign in the seat on Thursday, having concluded weeks ago it could not be won."
If this is true (and I have no reason to doubt it), this was a serious mistake. It suggests that UKIP may miss some opportunities next year because of lack of good local knowledge. UKIP have already released a list of target seats which looks eccentric in parts:
Aylesbury and Forest of Dean are not obviously in UKIP's top 12 best prospects (or even in their top 25). So far as I am aware, the wider list of 25 seats has not yet been made public.
This is important, because it means that we should downgrade UKIP's chances of taking seats unless, as in Clacton, they have good local ground information. The evidence that we have so far is that, Clacton aside, they don't.
What's next?
It is hard to overstate the importance of the Rochester & Strood by-election for understanding UKIP's chances next May. Douglas Carswell was in many ways the perfect defection for UKIP - a locally popular MP with a wider intellectual profile in a constituency with a ready audience for their message, able to take his team with him and to supply the ground game.
Rochester & Strood is much more typical, and will provide a good test bed both for future defections and for judging the reliability of constituency polls in Conservative-held seats challenged by UKIP. Mark Reckless has a fairly undistinguished record in Parliament and does not seem to be a particularly popular constituency MP. He had no wider profile until he defected. If he wins well, the Conservatives should be very afraid indeed.
What are my expectations? So far, we have only had a Survation poll, though I expect that we shall have one from Lord Ashcroft soon enough. On past reckoning, we might expect UKIP to be a bit overstated, the Conservatives to be a bit understated and Labour to be overstated. If this is correct, the current state of play is a real cliffhanger.
UKIP are likely to be given a boost by their success in Clacton (success breeds success), and their odds will certainly shorten in the seat. Will the Conservatives be able, as they apparently did in Newark, to squeeze Labour voters? Will Labour voters simply sit on their hands, or will they break in favour of UKIP? The Conservatives are certainly going to be going in very hard indeed, as they did in Newark. Labour seem to have decided consciously not to try.
I could see this seat being decided by a few hundred votes either way as matters stand right now (with quite a wide span of results beyond that too). UKIP should be favourites, but not overwhelmingly so. The Conservatives look value at anything over 2/1 to me, so I'm laying UKIP this morning. But I won't be betting too committally on this by-election: I'll primarily be waiting for its result. Once we have that, I will make what I hope are considerably more secure assumptions about how UKIP will do in 2015. It's unheroic, but prudent. Sometimes not betting is the wisest course of action. Given my track record on the trends with UKIP, it seems wise to me.
If you have strong views either way, conversely, you should be acting now. If you think that UKIP are going to win well in Rochester & Strood, you should be backing UKIP in most of the seats in which they are short-priced and challenging the Conservatives. If you think that the Conservatives are going to fillet the kippers in that by-election, do the opposite.
One final thought: unless you have inside information, betting on UKIP in seats on the basis that the Conservative incumbent is a possible defector is not for the faint-hearted. I'm not playing that game - in general it looks like a sure-fire way to lose money. I do note, however, that UKIP are 20/1 in Mid Bedfordshire...
That's one bet I'm not on though.