ChatGPT can’t beat human smart contract auditors yet: OpenZeppelin’s Ethernaut challenges
While ChatGPT-4 can’t compete with human auditors yet, OpenZeppelin noted it was not optimized to do so, and AI models trained for this purpose would likely be more accurate. While generative artificial intelligence (AI) is capable of doing a vast variety of tasks, OpenAI’s ChatGPT-4 is currently unable to audit smart contracts as effectively as human auditors, according to recent testing. In an effort to determine whether AI tools could replace human auditors, blockchain security firm OpenZeppelin’s Mariko Wakabayashi and Felix Wegener pitted ChatGPT-4 against the firm’s Ethernaut security challenge. Although the AI model passed a majority of the levels, it struggled with newer ones introduced after its September 2021 training data cutoff date, as the plugin enabling web connectivity was not included in the test. Ethernaut is a wargame played within the Ethereum Virtual Machine consisting of 28 smart contracts — or levels — to be hacked. In other words, levels are completed once the...