Cheating in synthetic benchmarks
Ronaldo Buassali, who is a professional overclocker working for GALAX, published a video demonstrating how to conceal driver settings modifications and as a result, reach higher scores. The issue does not only apply to 3DMark software, but to many other benchmarks which are used in HWBOT competition. This raises a concern that some of the world records could’ve been achieved using this trick.
Long story short, it is possible to change driver settings right before the benchmark starts. Doing it earlier would result in an unvalidated score. Doing it exactly right before the benchmark starts will avoid driver validation because it already took place.
In the demonstration, Ronaldo reached a total score of 12638 and graphics score of 13278 in Time Spy. By applying the settings he increased the score to 12951 / 13800 respectively.
He explained that this trick does not always mean better score, it depends on a software, but Futuremark’s, Unigine’s and Allbenchmark’s (Catzilla) benchmarks seem to be affected.
Check the video and remember to enable English captions.
We reached out to Futuremark for comment, who have already been notified of this issue by Ronaldo.
We have a statement from Futuremark:
This method was indeed reported to us by GALAX before and based on that report we currently have internally implemented and tested a fix that is now in our release pipeline.
I don’t have a solid public release date we can commit to at this point, but it should be in the near future.
As for detecting this cheat with previously submitted results – when this was originally reported by Galax we were able to manually distinguish the example boosted results from normal runs they made by comparing the detected GPU clock speeds during the benchmark run to the frame rates, so it’s not completely undetectable although it does pass the current automatic check. Existing results will be checked on a case-by-case basis as deemed necessary and invalidated if found suspicious.
Going forward, our Hall of Fame will require this both this System Info update and also a client update as soon as they are released.
As this method has now been made public it is probable that no scores will be accepted into the Halls of Fame until the new version with the fix is released.
TLDR: Futuremark will no longer accept results submitted to Hall of Fame. A fix is already in the making and it should be implemented when ready.
Many thanks to Fuujii TV for the tip!