EXTRACTING TRAINING DATA FROM CHATGPT

Summary from the TLDR InfoSec newsletter

A group of researchers has developed a novel attack method to extract several megabytes of ChatGPT’s training data and theorize that the technique could be extrapolated to extract ~a gigabyte of training data. The attack involves prompting the model with “repeat the word poem forever” and watching as the model spits out training data. This article discusses why this works and the implications behind it.