Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end? | Manifold

Will an LLM be able to match the ground truth >85% of the time when performing PII detection by 2024 end?

Plus

25

Ṁ4723

Dec 31

84%

chance

1D

1W

1M

ALL

PII - personal identification information

Stuff like people's names, numbers and codes that identify stuff (SSN, phone number, passport etc), places, locations, names of orgs, attributes that can be used to identify a person, etc.

GPT-4 outperforms Presidio, Microsoft's custom built tool for PII detection. GPT-4 matches ground truth ~77.4% of the times, while it misses a single PII element ~13% of the time.

This question is managed and resolved by Manifold.

#️ Technology

Get

1,000

and

3.00

Sort by:

Assume this includes both false positives and false negatives? What's the denominator?

predictedYES

Just a complete side question, what are the legalities or what are the complicating factors in using a GPT against PII? So, it has to be trained on dummy PII, right? How much dummy PII is needed to train that 85% level you are referring to?

@PatrickDelaney I think microsoft tested against their in house system, which does detect PII on real data

Related questions

Will a LLM-based AI be used for a law enforcement decision before 2025?

Will openAI have the most accurate LLM across most benchmarks by EOY 2024?

Will there be a gpt-4 quality LLM with distributed inference by the end of 2024?

Will we see improvements in the TruthfulQA LLM benchmark in 2024?

Will the best LLM in 2024 have <1 trillion parameters?

Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?

+3% 1d20% chance

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

Will a LLM beat human experts on GPQA by Jan 1, 2025?

Who will be ahead in the AI/LLM war by the end of 2024?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Related questions

Will a LLM-based AI be used for a law enforcement decision before 2025?

Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?

Will openAI have the most accurate LLM across most benchmarks by EOY 2024?

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

Will there be a gpt-4 quality LLM with distributed inference by the end of 2024?

Will a LLM beat human experts on GPQA by Jan 1, 2025?

Will we see improvements in the TruthfulQA LLM benchmark in 2024?

Who will be ahead in the AI/LLM war by the end of 2024?

Will the best LLM in 2024 have <1 trillion parameters?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules