1. ACTIS Insulation Ltd (ACTIS) submitted the BM TRADA Registration Certificate, covering the use of TRISO-SUPER 10 (TS10) in roof construction, and the BM TRADA Q-Mark Registration Schedule for TS10, which both stated that when TS10 was used as specified in the manufacturers' 'Fixing Instructions', its thermal insulation value was equivalent to mineral wool (glass) of 210mm thickness. They said the BM TRADA Certificate was granted on the basis of data in a report compiled by TRADA Technology Ltd entitled 'Summary report on the testing of ACTIS TS10 insulation between February 2006 and April 2006', which they also submitted. It explained how two identical chalet buildings, one containing TS10 insulation and one containing 200 mm thick mineral wool, were tested simultaneously in High Wycombe by monitoring the energy required to maintain their internal temperature at a set level. ACTIS said they had also derived data from a site in Limoux, France, and sent information on that.
ACTIS said that, in July 2011, TRADA's in-situ testing and neural network modelling (the Q-Mark Scheme) was approved and accredited by the United Kingdom Accreditation Service (UKAS); they sent documents to confirm that. They believed that proved the testing was sufficiently robust and that their claims were accurate.
2. ACTIS believed that in-situ testing, which measured the performance of insulation when installed in a manner representative of real buildings 'as built' and subject to actual weather conditions, was a more accurate and representative testing method than hot-box testing, which measured performance in static laboratory conditions devoid of weather conditions such as ventilation and wind penetration and which could not, and was not designed to, assess the ability of a product to perform as intended once installed. They submitted several reports from various sources which they believed supported their view.
They said the hot-box test was neither accurate nor representative and was currently being reviewed. The British Board of Agrément had approached ACTIS and other manufacturers with a view to designing a new, 'dynamic' hot-box test which took into account air flow and ventilation. They added, however, that those ideas were in their infancy and it was not yet possible to comment on the tests or their anticipated results.
Despite submitting evidence to support the claim that in-situ testing gave a more accurate measurement than the hot-box method, ACTIS also argued that the claim was not presented as accepted fact. They said the ad suggested that in-situ testing was an alternative method of testing, which was believed by ACTIS to be more accurate than the hot-box method. They said the ad was targeted at building industry professionals, who could be expected to know the basic factual circumstances surrounding in-situ testing and multifoil products. They argued that, furthermore, the recommendation that approval should be sought from the local Building Control body before installing the TS10 product clearly implied that the in situ test method was not yet accepted by Building Control, reducing the likelihood of readers inferring that the accuracy of in-situ testing over the hot-box method was an established fact.
1. Upheld
The ASA sought expert advice. The expert said, if the claim was accurate, that meant the TS10 multifoil insulation was performing three times better than he would have expected from well-established testing and standardisation procedures.
The expert obtained, from his own sources, the full report on the testing of ACTIS TS10 insulation, which went into more detail than the summary report. He said the test chalets were well designed and constructed and the instrumentation and data recording techniques were appropriate. The results suggested that the chalet with walls and roof lined with TS10 used 19.1% more energy than the one similarly lined with 200 mm (not 210 mm as mentioned in the ad) of mineral wool. The expert said it would be expected from the current established standards that the TS10 chalet would use more energy than the mineral wool chalet but, by passing the 19.1% result through a neural network model, ACTIS had transformed the test results into the claim that TS10 was thermally equivalent to 210 mm of mineral wool, without providing a clear and transparent account of the process by which that was done.
The expert considered that, although the neural network model, which was developed by an acknowledged expert in the field, was robust and appropriate to use in the analysis, it was not valid outside the range of the data used to develop it (the learning data). He was of the view that ACTIS had not demonstrated that the model, developed with learning data from High Wycombe, could be used to give reliable predictions of the long-term energy use in other locations.
The expert noted the data from meteorological stations, in more exposed and windier locations than the High Wycombe site, was filtered to remove that which was not within the range of the learning data, before the data was used to predict the consumption in the two test chalets. For example, if a single wind speed from one of the UK locations exceeded the maximum value from the learning data, that value and the corresponding temperature and solar radiation data would be eliminated, leaving a gap in the record. The expert said filtering climate data was not a simple process and could lead to unreliable results, because of the way in which climate parameters interacted with a building to determine its energy use. He argued that, for example, the temperature in any hour was strongly related to the temperature at the previous hour; there was a sequence of smoothly changing values, not random, unrelated values. Also, individual parameters were interdependent: for example, the weather tended to be colder in an easterly wind than a westerly one; and overnight temperatures were lower in low wind speeds than high wind speeds. Moreover, the energy demand of buildings depended on the combination of parameters, not the individual parameters (high wind speeds and low temperatures led to high energy demand). The expert advised that filtering out, from the meteorological stations data, values that lay outside the scope of the learning data, would disrupt the time sequences, the inter-relationships between the parameters and the consequent response of the building.
The expert said the information submitted by ACTIS showed that the agreement between the predicted and measured consumption in the High Wycombe mineral wool chalet was poor. Although the Limoux tests, like those at the High Wycombe site, were well designed, calibrated and carried out, the expert said the results of those tests were not processed properly, because again the agreement between the measured and simulated data was not good. There was therefore no evidence that the model did actually predict the chalet consumption accurately, and the expert said that meant he could not have confidence in the model's predictions for a wide range of other UK locations with a much wider range of climates.
The expert advised that, although the chalet tests were well conducted, ACTIS's interpretation of the results was unjustified and flawed, according to the information they had provided.
We expected ACTIS to have rigorous standards of evidence to prove their claim, but considered that they had not provided sufficient information to demonstrate how they had translated the test results - which suggested that the heat lost from TS10 was greater than that lost from the mineral wool - into the claim that TS10's thermal efficiency was equivalent to 210 mm of mineral wool. We noted our expert's concerns that the results obtained from the neural network model could not reliably be generalised to other buildings in other locations, and that uncertainties had been introduced by the filtering of the climate data before input to the neural network model. We concluded that the test results could not be interpreted as substantiating the claim that the thermal efficiency of TS10 was equivalent to 210 mm of mineral wool, and the claim was likely to mislead.
We noted that, in July 2011, TRADA's in-situ testing and neural network modelling (the Q-Mark Scheme) was approved and accredited by UKAS. However, we noted that it was not accredited or approved at the time the testing sent to support the claims was undertaken. We also noted that, as part of the accreditation, it was suggested that some amendments were made to the model, including increasing the minimum duration of the test period to three months, rather than the one month used in the testing sent to us as evidence. Regardless of the accreditation, we were also concerned, for the reasons stated above, that the test results could not be used to substantiate the claim.
2. Upheld
We sought expert advice. Our expert said there had been considerable discussion within the standardisation and research communities about the relative merits of laboratory and in-situ testing of the thermal properties of insulation materials. He advised that in-situ testing might have some advantages and was sensible in principle: if well conducted and analysed, it would provide accurate and realistic data on the performance of insulation in buildings under test. However, it was difficult to carry out and the tests needed to be rigorously designed and analysed to avoid false conclusions being drawn. Because in-situ testing was dependent on the specific building and climatic conditions of the test, it could be difficult to apply the results to other buildings or locations, whereas laboratory testing provided a benchmark value for a product, which could be corrected for specific circumstances of use with standardised correction procedures.
The expert noted ACTIS maintained that laboratory hot-box testing of their multifoils did not take account of heat transfer by radiation, thereby underestimating their products' performance. The expert pointed out that hot-box tests carried out by the National Physical Laboratory (NPL) in 2004 on the TRI-ISO SUPER 9 (TS9) product (which the expert believed had a similar thermal performance to the TS10 product) were designed to take full account of the effects of radiation and gave very similar results to those obtained by in-situ testing of the TS9 product at the ACTIS test site in Limoux, France in 1997 and reported by TRADA Technology Ltd. The expert was therefore of the view that, if carried out rigorously and analysed correctly, hot-box and in-situ testing would give similar results.
The expert also noted ACTIS's argument that the constant temperatures in laboratory tests did not reflect the dynamic conditions in the real world, and the effects of solar radiation and wind on real building fabric were not taken into account in laboratory tests. He said methods existed which allowed the results of laboratory tests to be applied to real buildings containing thermal bridging and imperfect workmanship. He also pointed out that TS10 was air tight and therefore found it difficult to understand why ACTIS placed so much emphasis on air flow and ventilation.
He agreed with ACTIS that according to the scientific definition, accuracy was not the same as precision, and understood that the words "more accurate" in the ad were intended to mean 'more realistic'. He also accepted that the words "the actual performance" in the ad were intended to mean 'the real life performance'. Nevertheless, he thought the reports ACTIS had submitted did not constitute evidence that in-situ testing was a more representative, more realistic testing method than hot-box testing, because it was very difficult to generalise the results of in-situ testing to other buildings in other locations.
He continued that European Standards, which were adopted as British Standards in the UK, on the performance of thermal insulation were developed by the European Committee for Standardization (CEN) Technical Committee TC89; their current position was that laboratory tests gave a reliable and realistic measure of the performance of multifoils. There were currently no standards for in-situ testing. However, a CEN workshop had been set up to develop recommendations for in-situ testing of multifoil products. CEN Technical Committee TC89 had been asked to start a new working group to develop standards for in-situ testing, using the draft report of the CEN workshop as the basis of its work. However, even if such standards were developed, they would not necessarily replace laboratory test methods.
We understood from the expert that the relative merits of laboratory and in-situ testing of the thermal properties of insulation materials had been the subject of much discussion within the standardisation and research communities. We noted the current position of the CEN Technical Committee TC89 was that laboratory tests gave a realistic measure of the performance of multifoils.
Although ACTIS argued that the claim was presented as their view, not as accepted fact, we noted they had submitted evidence in support of it, thereby treating it as a factual claim. We considered that readers would infer that the claim was capable of substantiation and was not merely an opinion backed by no facts, and were likely to expect ACTIS to have strong, robust evidence to support the claim. However, our expert's view was that the evidence ACTIS had submitted did not prove their claim. We also understood that the one available comparison of in situ and hot-box measurements showed very similar results for both methods of testing.
We concluded that ACTIS had not proved that in-situ testing gave a more accurate measurement of the actual performance of multifoil products than the guarded hot-box method, and the claim was likely to mislead.
On points 1 and 2, the ad breached CAP Code (Edition 11) clauses 3.1 (Substantiation) and 7.1 (Truthfulness).
