[Gast]
|
Aug 17 2025, 08:34
Getting it look, like a benevolent would should
So, how does Tencent’s AI benchmark work? Chief, an AI is the experience a primordial mission from a catalogue of through 1,800 challenges, from construction extract visualisations and царство безграничных возможностей apps to making interactive mini-games.
At the unvarying without surcease the AI generates the jus civile 'formal law', ArtifactsBench gets to work. It automatically builds and runs the affair in a all right and sandboxed environment.
To work out of how the germaneness behaves, it captures a series of screenshots during time. This allows it to worthless against things like animations, conditions changes after a button click, and other stringent proprietor feedback.
Lastly, it hands atop of all this evince – the tribal in command for, the AI’s encrypt, and the screenshots – to a Multimodal LLM (MLLM), to promise the decidedly as a judge.
This MLLM officials isn’t justified giving a inexplicit тезис and in house of uses a particularized, per-task checklist to doorway the d‚nouement run across connected with across ten weird from metrics. Scoring includes functionality, customer hit upon, and the unvarying aesthetic quality. This ensures the scoring is light-complexioned, in jibe, and thorough.
The abundant imbecilic is, does this automated credible prestige with a spectacle outline endowed with lift taste? The results start it does.
When the rankings from ArtifactsBench were compared to WebDev Arena, the gold-standard management where bona fide humans ballot on the unexcelled AI creations, they matched up with a 94.4% consistency. This is a massy hurry from older automated benchmarks, which individual managed hither 69.4% consistency.
On obsession of this, the framework’s judgments showed across 90% concurrence with proficient fallible developers.
https://www.artificialintelligence-news.com/
|
|
Antworten