Sleuthing solar: How data reveals invisible power generation
Posted: September 16, 2026
This year, as Britain faced its sunniest July ever recorded, some of the country’s solar projects were being switched off. The high temperatures soaring across Europe risked overloading grids across the continent as increased demand and the outdoor temperature impacted grid conductors and transformers. In Britain, the Derril Water Solar Park, the U.K.’s largest community-owned solar farm, had production curtailed, reportedly due to the large amount of rooftop solar in the area.
Even output itself is harder to manage in the heat: Perhaps counterintuitively, the best days for solar production are in the cool spring. High temperatures have a big impact on the management of the transmission network and air conditioning usage increases electricity demand. For PV panels, hot weather reduces the efficiency of power conversion, reducing output by around 0.5% for every 1℃ increase in temperature. This year’s cooler spring months had a peak generation of around 16.5 GW, while in the hot summer months peak generation fell to 15.5 GW, despite the fact that capacity had grown by 1 GW.
High solar or wind generation, combined with local network limitations, can make balancing supply and demand complex. Energy operators need clear visibility into the entire energy system to balance the grid in the most effective way. And solar power generation is harder to see and track than other renewables.
Britain’s solar capacity ranges from domestic rooftop arrays to large farms, and is spread across thousands of locations. It now has more than 2.1 million solar installations, with a capacity of 24GW. But only around 60% of Britain’s solar power comes from utility-scale sites. These factors make it more complex to track, monitor and curtail than, for example, wind energy, which while also intermittent, is mostly contained in large transmission-connected farms, meaning it’s visible to the system operator.
Professor Alastair Buckley, a University of Sheffield researcher who specializes in solar PV and energy systems, spoke to Our Industrial Life about how data is helping make solar more visible to the network, and what needs to change as the grid evolves.
The problem with tracking solar energy
For one of these complexities of solar power, there’s a solution. Professor Buckley first realized that there was a problem tracking solar energy about a decade ago. With thousands of small domestic solar panels installed, the National Energy System Operator (NESO)—the operator responsible for balancing Britain’s grid—had no way of knowing how much energy solar was feeding into the grid. From his position at the University of Sheffield, Buckley started a central, crowd-sourced database that gathered data on the combinations of solar panels being used throughout the country. The Microgen Database asked users to submit their own data—Buckley initially recruited users by putting leaflets through doors of homes with visible PV systems—and then provided analysis of how their systems compared to others.
Following a meeting with the energy forecasting team at National Grid[1], Professor Buckley realized there was an opportunity to scale the system to help NESO predict supply and demand at any time on a given day. The Sheffield team planned to mine its Microgen Database to get accurate generation data from a sample of PV systems for a given half hour—the period that the system operator works in—and then scale it up to give a representative real-time PV performance across the country. Microgen initially had data on about 1,000 solar systems, but the team realized it needed more PV system data and a wider spread across the country to be truly useful.
What is the future of solar tracking?
PV_Live is the answer to the grid’s decade-old blind spot. The University of Sheffield’s data service provides NESO with half hourly measurements of national and regional solar generation for the 1.5 million systems that are not actively connected and managed as part of the balancing mechanism. The data comes from smart home company Passiv UK, which shares data from its portfolio of PV systems, and PV output, a website that has crowd-sourced data on 80 million domestic and commercial systems around the globe.
PV_Live gathers key data—such as capacity, age and location—to help estimate the output of each solar system. Location data is vital, as it impacts where the power feeds into the larger electricity network. The orientation of the system and inclination angle are also important, but not all systems have this information, so statistical sampling is used to estimate missing data. Finally, the efficiency of systems reduces over time, so installation date and technology class data is gathered if it’s available. Today, PV_Live uses data from about 20,000 systems around Britain to estimate the output of the entire fleet of systems. The sample of systems is optimized to provide the best geographic representation of the installed fleet across the country.
Professor Buckley needed real-time data with resolution much higher than half an hour to cope with data outages and gap fill for the missing data. “This was the biggest technical challenge—to get an outturn estimate as close as possible to the end of the half hour period of interest,” Buckley explained. Passiv UK provides sub-five-minute data to enable this, and the Sheffield team worked closely with it to develop a reliable data feed for the purpose. The system now provides near real-time estimates at half hour intervals. These estimates help save consumers money by eliminating much of the uncertainty of solar PV generation, meaning operators need to retain less reserve generation.
Despite this careful balancing, there are still challenges with the data that need to be accounted for. Professor Buckley explained: “There are biases in our data feed. We don’t have as many large-scale systems as we would like, and we have to assume that the yield of roof-mounted systems are the same as ground-mount systems, which they are not. Ground-mount systems operate cooler than roof-mount systems as there is more space around the modules for cooling. So we underestimate the output of the fleet by a few percent.”
Building a greener grid in the future
PV_Live helped to solve one big constraint on the grid, its visibility problem. But as batteries and heat pumps become more popular, the system will likely face a new generation of ‘invisible’ energy assets and shifting demand challenges. To keep the grid resilient as electricity starts to dominate power generation, Buckley points out that the country needs “whole system planning, reforms on connections management and market developments for flexibility and operability.”
For all the ingenuity of PV_Live, the fact remains that distributed solar is a headache for grid operators—a small number of big solar farms would be easier for the grid to manage. But large solar projects are unpopular with the British public. “From a policy point of view, that goes down rather badly. There are land use conflicts and the public don't like solar farms as much—they think residential and commercial rooftop makes more sense. But actually from a system point of view, it makes more sense to have fewer numbers of larger plants,” Buckley said.
One way of quietly disincentivizing distributed solar, and encouraging larger developments, would be to reform Britain's electricity pricing system. Britain employs marginal pricing, meaning consumers always pay for the most expensive energy that goes into any electricity mix—which is almost always gas due to the limitations of green grid connections. Buckley explained, “Solar gets paid at the same price of gas, even though it costs nothing to generate at that point in time, which is what makes it such an attractive investment opportunity.”
A system that took into account the cost of generation, for instance, could bring prices down for everyone, which would reduce both the savings and income generated by distributed solar. “We need to completely change the way the market operates so we're not locked into a pricing system that is dominated by the most expensive generator… If you were to completely change the market, you would completely change the business case for investing in solar,” Buckley concludes.
For governments and regulators, it comes down to a careful balancing act between the bills consumers pay now and building a grid with more green electricity for the future.
