ChatGPT’s artificial intelligence generated dialog has become quite sophisticated – to the point where it can write convincing-sounding essays. So Edward Tian, a computer science student at Princeton, built an app called GPTZero that can “quickly and efficiently” flag whether an essay was written by a person or by ChatGPT.
I spent New Year’s Eve building GPTZero – an app that can quickly and efficiently detect whether an essay was written by ChatGPT or by humans
— Edward Tian (@edward_the6) January 3, 2023
In a series of recent tweets, Tian provided examples of GPTZero in progress; The app determined that John McPhee’s New York essay “Frame of Reference” should be written by a human and a LinkedIn post should be created by a bot. On Twitter, he said he created the app over the holidays and was motivated by the increasing possibility of AI plagiarism.
On January 3rd, Tian tweeted that GPTZero was not working, probably due to a larger than expected amount of web traffic. in one Substack Newsletter Tian announced today that more than 10,000 people have tested the publicly available version of GPTZero on Steamlit. (Both at the time of writing gptzero.me and the Streamlit version show errors, likely due to traffic volume.) In the newsletter, Tian said he updated the GPTZero model to “significantly reduce the rate of false alarms and improve output results.”
GPTZero uses perplexity and burstiness to determine if a passage was written by a bot. Perplexity is how random the text in a sentence is and whether the way a sentence is constructed is unusual or surprising for the app. Burstiness compares these sentences to each other and determines their equality. Human writing has more burstiness — meaning we tend to write with more sentence variation.
Since the launch of OpenAI, there have been many concerns about plagiarism ChatGPT on November 30, 2022. More than a million people used it within five days of launch. The AI-powered software can write simple essays and mimic the style of established authors. You can instruct ChatGPT to, for example, copy Shakespeare’s voice or write in the style of a New York essayist. There are snags in execution, but the results are noticeable with the right style. It’s not hard to get the AI to write an English-style high school essay and find the result pretty indistinguishable from a student-written assignment. That said, there are still limitations on what it can do. It is easy confused by riddlesand sometimes only makes facts. StackOverflow also banned any coding feedback generated by ChatGPT thanks to the frequency of bugs.
In December, OpenAI announced it would “watermark” the ChatGPT output to combat plagiarism.
In his NewsletterTian said he is working on more updates to GPTZero, including “improving model features and fully scaling the app.”
