• Refactored.io
  • Posts
  • OpenAI's classifier to identify AI-written Text

OpenAI's classifier to identify AI-written Text

Plus: Instagram founders launch new social media

February 01, 2023

Refactored.io



Good Evening. Meta’s stock is up 17% after better than expected revenue numbers for fourth quarter. Good day if you bought calls ;)

In today's edition:

  1. OpenAI’s tool to identify AI-written text

Today's reading time is 2 minutes.

Main Thing

OpenAI releases classifier to identify AI-Written Text

Just one problem, it doesn’t work.

What: OpenAI released a new classifier to distinguish between text written by a human and text written by AI.

Sounds like something useful until you see the numbers -

  1. True positive rate is only 26% - that means if the text was written by an AI, the classifier correctly identified it as AI-Written only 26% of the times.

  2. False positive rate is 9% - that means if the text was written by a human, the classifier incorrectly labeled it as AI-Written 9% of the times.

Zoom Out: Several universities and school districts have already banned ChatGPT. Use of classifiers to detect AI-written text is almost guaranteed, similar to plagiarism-detection tools used today.

Imagine a classroom of 100 students, where each student completed their text assignments honestly without help of an AI. This classifier will still flag 9 of those submissions as AI-written. Guess who takes the blame?

An inaccurate classifier with high false positive rates will do more harm than good.

Big Picture

  • Instagram co-founders are launching a new social-media called Artifact, for news content. The founders want to use TikTok style algorithm to recommend news articles

  • DOJ is investigating Tesla's AutoPilot to find if the Tesla’s design and capability claims gave drivers a false sense of security. DOJ is looking into data from over 830,000 teslas equipped with autopilot.

Quick Links