Science or snake oil?

Science or snake oil?

This feature originally appeared in issue 14 of Nutmeg the Scottish football periodical.

Science Or Snake Oil? From Nutmeg 14 © Christoph Biermann

It is a fact of youth that every generation believes themselves to have invented sex, drugs and rock and roll, but the abiding impression on reading Christoph Biermann’s Football Hackers The Science and Art of a Data Revolution is that the current crop of bright young things consider themselves to have also invented football.

From his base in Berlin the German journalist wrote eight years ago: ‘Data is part of an ongoing evolution that changes football from a game of opinions into one of knowledge.’

It is a bold claim, not without foundation, but the question remains who is this so-called data revolution for and can it materially improve both the quality of our beautiful game and the conversation that surrounds it?

Presenting what is an undoubted meisterwerk, a ‘where we are now’ of football’s first world, the German writer says his book is about ‘the possibility of changing and improving the way of thinking about the game, for us as fans but also club officials,’ as he claims ‘football’s hackers are here to stay’.

For those working within the game, the impressively assembled anecdotes are fascinating but the book will be short on revelations.

Shots from 12 yards are more likely to lead to goals than headers, as are shots from counter attacks, when the opponent’s defence is caught in a state of disorganisation. Setpieces provide their own scoring probabilities too. But this is hardly revelatory stuff. There are no surprises here and the generalisations inherent in the big picture analysis are really of much less practical benefit than a first principles scheme designed to make best use of the players at a manager’s disposal.

For example, Football Hackers presents Smartsodds’ Matthew Benham’s assertion that he’d advised staff at FC Midtjylland that it was statistically preferable in terms of outcomes to pursue a second goal when leading, as a significant finding. Yet any current league manager in Scotland would not be at all surprised.

Indeed Billy Stark and his successor Gardner Speirs at lowly Queens Park made it a central plank of their approach, likely informed by Sir Alex Ferguson and Archie Knox. Both said that teams that fail to achieve their objectives over a season are likely to have sustained too many draws. The idea of playing on the front foot right up to the final whistle, as a form of game management, was fundamental to a successful era for Queens Park. And its legacy is most evident currently, in Scotland, in the never say day approach adopted by another QP man Stuart Kettlewell as co-manager at Ross County.

I don’t know Kettlewell but its clear that his team play the percentages, being prepared to take the odd heavy defeat in the act of securing enough points to guarantee’s safety in The Scottish Premiership for another season.

It is a similar story with Jurgen Klopp’s Heavy Metal football that looked so fresh back in 2010 or Man City’s ‘free eights’ midfield in Pep Guardiola’s early Man City career.

As former Scotland, Aberdeen and Man United goalkeeper Jim Leighton explains: “Everything say, that Liverpool are doing now, how they press the ball to win back possession, their relentless tempo, their commitment to dynamic, attacking football, Aberdeen were doing all that over 35 years ago.”
And as World Soccer pointed out at the time, Guardiola’s initial Man City midfield evoked the shape adopted by Queens Park’s cup winning side that faced Celtic in March 1893. Fashion, even in football, is just the return of familiar clothes that have been languishing forgotten at the back of the wardrobe.

Pep Guardiola’s use of Free Eights at Man City in 2016 shows that there’s nothing new in football, just new ways of talking about old ideas © World Soccer

For football outsiders, one of the more appealing aspects of this global game lies in the fact that everyone is allowed an opinion. And this white collar-inspired revolt into data has wittingly created a battery of cheap off the peg opinions, for people who, with the best will in the world, would otherwise have little of value to say.

There’s nothing wrong with unaccountable opinions in principle. If the aim is creating water cooler moments, diverse social media discussion or even a new kind of football journalism. But the money lies elsewhere in this equation – in persuading curious but intellectually insecure football people (and the people that fund them) that they’ve had it all wrong all along and that there’s a new route that can save them money, give them a short cut to success and create a football business model that is analogous to measurable, predictable corporate life.

At its core, Football Hackers is really a story of the cynical smart mouths, sharp speculators and pious fellow travellers that have their eyes on the main chance of taking or making money from the game. At its best it is true believers with a dream. At its worst it is the logic of Silicon Valley writ large: the consolidation of power and capital disguised as democracy, freedom and convenience.

Here is what inside Nutmeg 15 © Nutmeg

But of course, it was ever thus. George Bernard Shaw’s assertion that: “Revolutions have never lightened the burden of tyranny: they have only shifted it to another shoulder,” is as true now as it was when he uttered these words in 1903.

But to be fair to Christoph Biermann, he doesn’t shy away from reporting anecdotes and opinions that do not serve him well as an evangelist for data science’s Nouvelle Vague.

He reports football’s sacred knowledge where he finds it like the veteran German manager and sporting director Jörg Schmadtke sending his scouts on assignments specifically unaware of the players he was interested in, so they’d just call it as they see it. Or indeed of the ex-Arsenal Head of Recruitment Sven Mislintat learning on the job at Borussia Dortmund under the wing of the grizzled veteran scouts he was nominally managing.

the ex-Arsenal Head of Recruitment Sven Mislintat is a poster boy for the data revolution, but he confirms learning on the job at Borussia Dortmund under the wing of the grizzled veteran scouts he was nominally managing ©

Biermann also casts doubt on the claims that data, once analysed, can be anything like a value-free panacea. He says: “Numbers can easily give rise to more cognitive bias. There’s a huge temptation to cherry-pick stats that confirm a pre-existing view of a player and to dismiss those that present a conflicting picture.”

And this is especially the case when players or data are being evaluated ‘by committee’ and decisions become the product of what the parties ‘dislike least’ rather than what the key decision maker would otherwise choose freely. It is no different when the heavy lifting is done algorithmically. An algorithm relies on a weighting of data to function and that weighting always betrays the prejudices and viewpoint of its author.

Christoph Biermann author of Football Hackers The Science and Art of a Data Revolution (Blink Publishing) © Pablo Castagnola/

Biermann, as I do myself, also worries about a way of working that by its very nature plugs gaps in knowledge with data, whether that data is significant or not.

He says: “How can you measure players that do nothing brilliantly?”, those players whose positioning and reading of the game creates opportunities for teammates by taking away defenders or averting danger, before it even occurs. How do you begin to talk about games with data where the eyes say the evidence is not significant?

There is also a danger of looking at case studies in football and assuming that their models can be firstly adapted or imported to another club or country. The author offers a timely reminder that football is ultimately a human pursuit when considering the lessons we could learn from the social proof fallacy of Pep Guardiola’s success.

He says: “In Barcelona, Munich and Manchester he has had the best players at his disposal. They had been mostly winning in the years before his arrival and continued mostly winning once he was gone. Those teams would have probably won a majority of games without any sophisticated tactical decisions, too.”

Bayern analyst Michael Niemeyer told Christoph Biermann that Pep Guardiola was not interested in data; he drew all the information he needed from videos.

Biermann says: “Pep Guardiola didn’t further the development of football because he had access to a laptop with unlimited game footage. He looked at those videos with a sharp eye, a deep understanding and a firm idea of what kind of football he wanted to play. And he saw what others didn’t.”

Guardiola of course, like almost every other manager gained his football education from elders, Johan Cruyff amongst them. I’ve been delighted to pick the brains of numerous managers and players I’ve known over the years, some I’ve worked with, others I’ve met at games or in social or work situations. Operating within a relatively new discipline as so-called data scouts, those at the vanguard of football’s data revolution are by definition desk-bound, mentored by peers and possessing opinions for which they are largely unaccountable – even in the field of recruitment.

I recently spoke to a well-known coach at a famous English league club who summed up the situation perfectly. He said: “You arrive with the manager at a new club and the situation is always the same. Everyone has a scapegoat. The outgoing manager says he didn’t get supported, the Director of Football says his relationship with the manager broke down. The recruitment department say that all the signings were the manager’s and that he didn’t take on board any of their recommendations. You never get to the bottom of things.”

There’s an element of corporate politics involved in data’s adoption. However, a sense that data analytics is not central to the piece, even at the big clubs, remains on all sides.

The analysts bemoan their lack of position and off the record managers will regularly confirm that their analysts are not really offering a resource they benefit from directly. One of the biggest bosses in the game confided to a contact that really a lot of club’s resources in analytics and sport science are hired as a source of reassurance for potential player recruits. He said: “Without this in place we would struggle to get players we want to sign. It is part of their expectation.”

Perhaps the most uncomfortable truth for football data’s true believers is that for all their efforts, this low scoring game generates results that are more a product of luck than judgement on any given day.

That is not to say that data analysts cannot supply a compelling descriptive narrative after the fact, and claim that its lessons might be applied successfully in future. It is just that the predictive powers of a pre-game analysis are at best disputed and at worst wholly unreliable when it comes to predicting individual games or a small sequences of matches – the sort of sequences that lead to players and managers being overhyped and also sacked.

The random element, fundamental to football on a match by match basis, is reflected in the fact that a long-term betting profit of 3%-5% ROI betting on football match results would be considered a world class performance, even at the likes of big multi-million pound industrial syndicates such as Smartodds or Starlizard.

Anyone that claims they can engineer out the random elements of football and people to create a predictive blueprint for success in recruitment or match analysis is selling snake oil. What this leaves you with is a grey area of potential benefits that may or may not be directly attributable to the work undertaken. It is a fiction that only prevails as long as everyone buys into the provider’s narrative.

One of the most surprising revelations of The Football Hackers is the fact that even those on the payroll remain unreconciled to a way of seeing football that is an article of faith for others. Even in the afterglow of Germany’s 2014 World Cup win in Brazil, their national team’s Head Analyst Christoph Clemens confided: “We’re increasingly convinced that there’s a lack of data that provides real information about the things that make you successful in football.”

Two years later, according to Biermann, if anything Clemens’ attitude had hardened. “All usable data only illustrates the game retrospectively,’ he said. ‘It describes what happened and is basically so superficial that we cannot make any predictions based on it. We cannot use it to draw up an idea for developing players in a specific way or to come up with concrete instructions. It tells us nothing about success or the impact of a strategy.”

On the screen, wide angle football footage looks like chess played by humans, from the scout’s seat it looks like an often vain attempt to oppose collective order on randomness, punctuated by moments of individual triumph and disaster. At the level of the touchline it looks like warfare, with the result conditioned by the outcomes of brutal, bloody physical and psychological battles all over the park – even at the highest level. There is one thing that a wide view and the ability to stop and start the action can’t convey. That is context. And in football context is everything.

The reality is that no two games are the same, no two formations function comparably and no two stadiums, surfaces or direct opponents can be modelled to create the sort of predictive outcomes you might expect say from studying mortgage default analysis or cancerous cells where the data is rich and consistent and causation is understood and broadly uncontested.

It is one thing quantifying something such as an over or under-performance relative to luck, chances created or xG but the reality is that these trends don’t exist in a vacuum because their emotional effects degrade and enhance confidence in real time. Any data swims within a context of opposing managers and players looking to expose opponent’s weakness and play up their teams strengths. It also exists within the context of people like me – next opponent analysts – who are tasked with picking apart the good, bad and ugly of rival teams and players in a way that can and does make a nonsense of general trends.

A player that jumps in the defensive wall, a right back that’s easily beaten down the line or a forward that can’t play with his back to goal are facts that can directly influence the next result – and identifying them is the name of the game for both sides.

So, one side looks like they are pressing half-heartedly and in isolation? Well, that can sometimes look the same as opponents trying to make sure that the weakest player in the back four gets the ball funnelled to him as much as possible. Some outputs are one or two games specific, relating to instructions or even luck. Some are embedded in the character and construction of a team. It is hard to separate the signal from the noise.

The England Rugby Head Coach Eddie Jones confronted the fundamental issue at stake here when he said in April 2017: “We need to change players from being reliant on structures to having the confidence to play unstructured. You’ve got to coach the players to play the game, which in the most part is chaotic.”

One former manager I’ve worked for confirms: “There’s only two stats that really count. The last result and the next one, because everything you do will be judged by the fans and your board based on those scorelines. That’s the reality.”

Despite fans’ lip service to backing their team and the boss, the longer term testimony of bad luck, injuries, the fluctuation of form and the fixture list are as nothing when set against results. The rule of thumb is that most manager’s are only a run of three or four bad games away from the slippery slope to the sack. The honeymoon period gets shorter and shorter.

My ex-manager says: “You wonder what chance have you got? Everyone is an expert now and you get the impression that clubs would change their manager every six weeks if they could.”

Statistical outliers of form, luck, injuries and results will inevitably regress to the mean of expectations. But that is no consolation for the manager who has already fallen foul of football’s brutal short-termism. What no-one can predict is exactly when and how equilibrium will be re-established.

In theory, studies like Colin Trainor’s thoughtful work for Statsbomb on the ill luck of late period Dortmund under Jurgen Klopp for example are well intentioned but probably irrelevant to the piece. Devil’s advocates typically come into play only after the guillotine has fallen on a manager or the die has been cast for particular players.

Christoph Biermann says that there ‘has rarely been an attempt to systematically evaluate whether a coach is lucky or unlucky before he is dismissed’. The reality is though that neither fans paying their money, nor suits protecting their behinds would want to entertain the possibility that luck offers any kind of mitigation.

Another reality is the information gap that exists between football’s observers and its insiders. The first group assumes much but are always surveying an incomplete picture whereas the latter knows much but reveals little.

Football, internally, is a game of closed networks and restricted codes of conduct and language both by choice and necessity. What football says publicly and what it does and believes privately are generally very different. And for solid, practical reasons. And this also extends to football’s internal body of knowledge.

Even in notoriously dark decades such as the 1980s there have been teams such as Aberdeen and Dundee United, Tottenham and Ipswich playing football ahead of their time. Football’s evolutionary history exists not as linear progress but as a series of peaks and plateaus, and largely as an oral history. Its stories are kept alive by each generation of insiders passing on their secrets to the next. And these are stories that the press and fans are not privy to.

The ex-Dundee United and Scotland Hall of fame inductee Maurice Malpas says that United’s legendary boss Jim McLean was years ahead of his time as his team took on the best teams in Europe season after season in the 1980s © Prep4Pro

Maurice Malpas the former Scotland and Dundee United fullback confirms that football’s relationship with technology and data is really nothing new.

He says: “Jim McLean probably had the biggest satellite dish in Tayside and when the first home video machines came out he’d obsessionally tape every game he could and show us relevant clips. It is a wonder he didn’t break the rewind button. When we played he’d also have a scout in the stand, taking note of what he called our ‘positive passes’ so he could confront us with the evidence later. We had dieticians in, sports scientists and psychologists at Tannadice at different times, anything Wee Jim thought might give us an edge.”

Maurice Malpas says: “The only real difference between then and now is the changed terminology and the slick way information is packaged and presented.”

The emergence of popular tactics websites and discussions has created the perception that the game has become more intricate, more cerebral but it is largely a function of presentation in the modern day. Virtually every innovation hailed as contemporary in football has an historical antecedent. No-one is reinventing football – regardless of what their called or the roles they perform.

Nutmeg have a 3 for 2 offer on all back copies (including Issue 15 for a limited period).
Go to this link Choose 3 editions. And they will be in post & dropping through your letter box before you know it.© Nutmeg