Maybe AI agents can be lawyers after all
Maybe AI agents can be lawyers after all
In Brief
Posted:
Last month, I wrote about Mercor’s new benchmark measuring AI agents’ capabilities on professional tasks like law and corporate analysis. At the time, the scores were pretty dismal, with every major lab scoring under 25%, so we concluded lawyers were safe from AI displacement, at least for now.
But AI capabilities can change a lot in a couple of weeks.
This week’s release of Anthropic’s Opus 4.6 shook up the leaderboards, with Anthropic’s new model scoring just shy of 30% in one-shot trials, and an average of 45% when given a few more cracks at the problem. Notably, the release included a bunch of new agentic features, including “agent swarms,” which may have helped with this kind of multistep problem-solving.
Regardless, the score is a huge jump from the previous state-of-the-art, and a sign that progress on foundation models isn’t slowing down. Mercor CEO Brendan Foody, who was particularly impressed, said, “jumping from 18.4% to 29.8% in a few months is insane.”
Thirty percent is still a long way from 100%, so it’s not like lawyers need to be worried about getting replaced by machines next week. But they should be a lot less confident than they were last month!
Newsletters
Subscribe for the industry’s biggest tech news
Related
Latest in AI
Sponsorizzato
Sponsorizzato
Sponsorizzato
Sponsorizzato
Pubblicità
Categorie
Leggi tutto
We had sex in a Chinese hotel, then found we had been broadcast to thousandsWanqing ZhangBBC Global China Unit and Eye InvestigationsBBCOne night in 2023, Eric was scrolling on a social media channel he regularly browsed for porn. Seconds into a video, he froze.He realised the couple he was watching - entering the room, setting down their bags, and later, having sex - was himself and his...
What the Super Bowl means to American fansSkip to contentWatch LiveBritish Broadcasting CorporationHomeNewsSportBusinessTechnologyHealthCultureArtsTravelEarthAudioVideoLiveDocumentariesWatch LiveMore on this story.3 hours agoSuper BowlUnited StatesCalifornia
CRA denies hard of hearing taxpayer disability creditThe Canadian Revenue Agency's national headquarters in Ottawa. Photo by Sean Kilpatrick/The Canadian Press filesArticle contentIf you have a severe hearing impairment, you may be entitled to claim the disability tax credit (DTC). The DTC is a non-refundable tax credit that is intended to recognize the impact of various non-itemizable...
What No One Tells You About Rebalancing Your Portfolio We research all brands listed and may earn a fee from our partners. Research and financial considerations may influence how brands are displayed. Not all brands are included. Learn more. Money is not a client of any investment adviser featured on this page. The information provided on this page is for educational...
How Some Investors Use Gold to Protect Their Savings During Market Shocks We research all brands listed and may earn a fee from our partners. Research and financial considerations may influence how brands are displayed. Not all brands are included. Learn more. Money is not a client of any investment adviser featured on this page. The information provided on this page...
