Why employment figures aren't up to the job

Unemployment in Scotland is down, while it has risen in the rest of the UK.

The monthly figures on the jobs market tell us the share of the adult population in Scotland searching for a job between March and May was down to 3.7%, while it was up to 4.7% across the UK

That seems like good news for the Scottish economy. But can these figures be trusted? According to the people who compile them no, they can't, and be careful how you use them.

The margin of error for the Scottish figure is 0.7 percentage points either way, so unemployment could be 3% or 4.4%. And while there's a smaller margin on the UK figure, it means that the Scottish figure could be higher than the UK one, but probably isn't.

Such numbers are no longer given the stamp of approval as National Statistics. Why? I'll come back to that.

But let's look at some numbers that are more reliable. They come from HM Revenue and Customs which - as many of us know - takes tax off employed people's pay at source in the month it is earned.

The collection agency knows the number of people on payrolls, how that number varies, and how median pay for employees is changing. They can do that at a Scottish level as well the UK and other bits of the UK.

In Scotland, that tells us there were more than 2.5 million people in payrolled jobs in June. And since June of last year there's been a fall in that total of 16,000. With more data due, they'll revise that figure next month.

Another reliable way of counting captures only one element of the number of people seeking work in any month - the claimant count of those seeking benefits due to unemployment. Lots of people who are unemployed do not claim or don't qualify for benefit.

That claimant count hasn't changed much over the past three years. The newly-published figure is 107,200 claimants in Scotland last month - up 1,800 on the previous month, and down 1,300 on June last year.

According to HMRC, median employee pay in the month to June was up 5.4% and earning £2,546. (A reminder: if all employees were lined up, the median would be the person in the middle.)

Across the UK, that figure rose faster, at 5.6%. And when you take inflation into account - at 3.6% we learned this week - real earnings rose by 1.8%.

That does not include self-employed workers. There are very poor stats on them until a long time later because they don't declare their income, or even if they've been working, until several months after the end of the financial year. Earnings this month have to be declared by the end of January 2027. The figures filed in January this year were for earnings up to 20 months before.

The other measure of earnings from the ONS is from a survey of British workers taken between March and May, including the self-employed but not including Northern Ireland (for reasons I haven't yet discovered).

That increase in average monthly earnings was 5% in cash terms. That's down from 5.3% in the previous three months, December to February. And if you exclude bonuses, it's the lowest rate of increased earnings for nearly three years, reflecting the fall also in price inflation.

If you're in the public sector, average monthly earnings were up 5.5% in the year to spring, in the private sector it was 4.9%.

The real increase, after accounting for price inflation up to those spring months, was again 1.8%. But if you include housing costs, only a 1% real boost to your spending power.

This takes us back into the murky statistics of the Labour Force Survey. It's carried out throughout the year by the ONS. It used to be reliable and statistically sound, but since the pandemic, things have gone badly wrong with it.

That's partly because survey response rates from the public have fallen sharply. It seems we're not as willing to engage with people asking us questions about our working lives and our earnings, particularly younger people.

It's also because the ONS embarked on a change in the way it gathers information, moving from face-to-face-interviews to online questionnaires. It planned that transition badly, and the stats that it gathered ceased to be reliable.

The ONS claims things are turning around. They're getting more people in the survey, down from 84,000 every three months to 44,000 when, to quote an ONS source, things were "truly awful". That's risen to nearly 70,000 people questioned.

So it's getting better, but still not certificated as sound. And don't bother comparing the most recent survey figures for March to May with last year, or the year before.

Also, if you dig into these figures, you'll find that big range of possible outcomes. For instance, the estimated increase in the number of Scots in employment in March to May was plus 22,000, but that could be wrong, by plus or minus 74,000. So there's a reasonable chance that the number fell.

Does it matter? Yes, because in making policy and distributing funds, those in government need to have the best information about the economy.

The Bank of England's chief economist complained to the ONS about the difficulty of setting interest rates when the employment figures cannot be trusted.

Or take the concern about the number of people who are not available for work, and classified as economically inactive. They're a big focus of government policy.

The benefits bill for people who are economically inactive due to long-term illness has been rising very steeply. The government came badly unstuck when it sought to cut entitlement to that welfare benefit, and was forced into a U-turn.

How do they know how many people are not able to work because of ill-health, or would like to work if they got more help, or have no intention of working? They look to the ONS figures.

The UK-wide figures are more reliable, because there's a much bigger sample. But even with the increased sample, the margin for error when taking only parts of the UK can leave much uncertainty.

Within the Scottish cohort in the most recent numbers, the number of economically inactive people is estimated to be 756,000, but that could be out in either direction by up to 68,000.

The ONS has done a full reversal of the move to online questionnaires, at least for the meantime. It plans a long-term shift back to using online questionnaires, while avoiding the mistakes made in the past few years.

For now, the survey is based on face-to-face interviews for the first encounter and five follow-up phone calls and online questionnaires with the same people in each of the next five quarters.

There have been other foul-ups in collecting the stats, including a hiatus in producer prices. In clearing up the mess at ONS, the boss went quietly in May, citing health reasons.

Sir Ian Diamond was previously principal of Aberdeen University, where his pay-off was the subject of national controversy.

An independent inquiry into the ONS was set up in April and reported last month, referring to big problems with the management style on Sir Ian's watch, where people were unable to challenge the workload of reform while continuing round-the-calendar demands.

The workload was unrealistic, bosses did not want to hear unwelcome news, the inquiry found it unsurprising that senior people were leaving, and the quality of output was declining.

Last week, ONS chair Sir Robert Chote quit his post, after being told by the government that new leadership was needed, along with integrity in the figures.

So there are changes being made to the way the ONS is run, with a recommendation from the inquiry report that an experienced change manager is brought in, while someone else leads on the crunching of numbers.