Rendered at 16:22:34 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Retr0id 1 hours ago [-]
Under what circumstances does the metadata get added? When Claude Code CLI needs a media file usually I just see it run an imagemagick or ffmpeg command to create one, which isn't going to have C2PA metadata.
Edit: I asked Opus 5 for "a jpg of a cat" via the web chat. It did it by writing an svg and then rendering that to a jpeg. Clicking the "download" button saved a jpg which did have C2PA metadata, seemingly injected by the artifact-presentation thing.
Then I told it to put it in a zip file. I downloaded the zip, and the extracted jpeg had no C2PA metadata.
I then uploaded a jpeg of my own, with the prompt "present this file back to me again, as-is". The downloaded file had C2PA metadata injected.
Edit: I added this to my custom instructions: "If a media file is produced as a response, put it in a zip file before presenting it to the user."
AmazingTurtle 35 minutes ago [-]
Well, guess they are just trying to conform all the weird EU checkmarks
csmoak 55 minutes ago [-]
As far as I can tell, this and the recent change to add watermarking to text outputs[1], is to become compliant with the EU AI Act[2] and CA's AI Transparency Act[3], SB-942[4]. For large enough companies, all generated AI content is required to have watermarking.
This is just C2PA for Claude-processed files, apparently. It's not related to their text watermarking.
C2PA is file metadata and can be trivially stripped away, unlike hidden watermarks, e.g. SynthID.
mhitza 58 minutes ago [-]
That is where their responsibility ends in terms of the EU AI Act. And that is fine and how it should be, no secret watermarks.
a3w 3 minutes ago [-]
Watermarks can even be removed, "if substantially proofread" in the EU, law says (for now. There might be a transition period on that).
For the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear if making watermarks removable is a may, shall, or should, according to legislation. It certainly is not a must.
a3w 4 minutes ago [-]
Watermarks cane even be removed, "if substantially proofread" in the EU, law says. For the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear is this is a may, shall, or should. It certainly is not a must.
a3w 6 minutes ago [-]
Watermarks should even be removable "if substantially proofread or altered" in the EU for the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear is this is a may, shall, or should. It certainly is not a must.
ramon156 1 hours ago [-]
The way I read it, I thought they would just check if the file hash exists on their end. Is that not the case?
dannyw 1 hours ago [-]
That’s incompatible with the file not leaving your browser, which you can trivially verify with the Network tab in chrome (or wireshack, etc).
This is just a C2PA metadata checker.
spockz 56 minutes ago [-]
Why? The browser can see the file and contents so it can calculate and send the hash over without the actual file ever leaving the browser.
quaintdev 1 hours ago [-]
> I thought they would just check if the file hash exists on their end
That would invalidate the hash on minor changes. Too much effort and not enough return.
atp_studio_coll 1 hours ago [-]
[flagged]
coffeecoders 15 minutes ago [-]
What is interesting to me is that stripping the C2PA data is easy, but faking it is hard.
You can resave the file and the "made with Claude" signal disappears, but you cannot make a random file pass as Claude-made without Anthropic's signing key. So the useful guarantee is one-way. No signature means almost nothing.
8cvor6j844qw_d6 4 minutes ago [-]
Interesting, removing from detection is doable but wondering if it's possible to induce a false positive.
firefoxd 2 hours ago [-]
This is really fascinating. Even the AI companies have incentives to reject AI generated content. It's like they want you to use AI for everything, but they don't want AI output fed back to them.
At work, right after an AI training, we were asked to use our "authentic" voice when writing mid year reviews.
stranded22 2 hours ago [-]
Just like the tech company owners not allowing their kids access to social media…
docjay 24 minutes ago [-]
Not really. This is more like not pumping the exhaust back into the intake.
hmokiguess 2 hours ago [-]
Could this be used to perform some sort of distillation or exploit? e.g. reminds me of the OWASP guideline on attack vectors where knowing if an ID is present or not in the database can be a form of exploit, like in password resets where they will say 'email foo@bar.com not found' rather than 'If foo@bar exists we have sent an email to foo@bar' or some other generic equivalent
kelvinjps10 2 hours ago [-]
I think all of this it's so they don't get ai generated content in their training data
bradfa 44 minutes ago [-]
Or it’s so they can continue to operate in the EU where this is required.
applicative 2 hours ago [-]
So they do it as a free service to the other LLMs?
moritzwarhier 1 hours ago [-]
They still scrape code, I'd guess, e.g. from GitHub?
And there's tons of Claude-generated code there.
Also, I'd guess this is not just to prevent any AI-generated code in the training data, but specifically their own.
Percentage of users who put out their code on the web and also have a plan where Anthropic promises not to train on their data is problem also low.
So not excluding own code could be a real issue, since it would be impossible to deduplicate the training and RILHF data from their sessions with the code accessible elsewhere, and written by the very same users.
I'm wondering why they have restricted file types. You can't check a PDF for example... surely the main use case for people will be to check if a document was produced or edited by an LLM? That could be an attractive (if not misunderstood) proposition for academics
spockz 57 minutes ago [-]
Luckily all pdfs I made in academia have been generated from source (tex or derivatives, asciidoc) and llm are better in generating source than pdf. Even people that didn’t use tex used Word to generate the pdf.
So I’m unsure what your aiming at.
rdtsc 21 minutes ago [-]
Source can have markers inserted in it as weighted word choice or phrase choices, how commas are inserted, etc., so that the output can still can be identified. In other words just because it's source doesn't mean it can be watermarked.
tom1337 52 minutes ago [-]
Unfortunately this is just for Media? Some manual tells for Excel or PDFs is to check the author. Claude creates PDFs via wkhtmltopdf so the PDF Producer will be Qt and the Content Creator is wkhtmltopdf. Xlsx files are being created via openpyxl so in the metadata that is the author.
kbrannigan 2 hours ago [-]
How long before they change the terms and conditions to subtly claim ownership of your files? When you write code they already insert Co author attribution/
Say I write a text by hand And then I tell it to clean up the grammar and fix some sentences Did it make it?
This is also interesting for those companies that siphoned the entire open web
warkdarrior 2 hours ago [-]
Microsoft Word has not claimed ownership in 40 years. Why would Anthropic do?
apetresc 7 minutes ago [-]
I'm so curious if you're actually able to keep a straight face while typing that.
transcriptase 1 hours ago [-]
Because Anthropic are the type of strange people who think that language models have feelings.
verdverm 2 hours ago [-]
They have said and tried some wild things already, like trying to get open weights effectively banned, which I believe they still think is in society's best interest (more that they think they know what's best for everyone)
htrp 1 hours ago [-]
The actual implementation matters less than the fact that they can be perceived to be working on something.
You're never going to have a perfect solution here, and perfect is the enemy of good.
dgellow 22 minutes ago [-]
> Knowing where content came from, and whether AI was involved, makes it easier to trust what you see online.
That’s a cute way to imply their service is used to generate misinformation. They are basically saying to not trust the AI content made from their own product :)
quinndupont 54 minutes ago [-]
I could do without more surveillance.
declan_roberts 49 minutes ago [-]
It's for your safety!
Surac 2 hours ago [-]
droped some source files from different languages and i could process non of them
Mashimo 1 hours ago [-]
It clearly tells you what files it can process. This is about media files, not source code.
shujip 2 hours ago [-]
A watermark answers "did this model touch the file?" It doesn't answer "did a person read it and stand behind it?"
Detection will get gamed, and a lot of useful text is already mixed. The more useful check, for me, is still human: would I put my name on this before it goes out?
derdaniel 2 hours ago [-]
this is going to be the big thing in the next 100 years for me. where there is a human, there will be context, passion and meaning. it's the old saying: machine can tell you what, but only humans can tell you why.
ahmadtbk 2 hours ago [-]
[dead]
VCFundedGenYer 2 hours ago [-]
Claude watermarks are a farce and a waste of time. It's hilarious to me that they burn cash to even entertain the idea.
Any attempts they use are defeated by a text editor and CTRL SHIFT V. Unicode characters are no new thing.
Reminds me of how people tried to argue that NFTs aren't anything more than just jpegs.
sente 2 hours ago [-]
You misunderstand how Claude watermarks things.
mr_mitm 2 hours ago [-]
TFA is talking about C2PA, a regular signature in the meta data. A lack thereof means nothing, but if it's there, the false positive rate should be near zero.
basedpolymer 1 hours ago [-]
Most critics of watermarking have no idea how it works. Yet there are valid arguments against (and for) it, but they prefer to be misleading
dgellow 19 minutes ago [-]
What are the valid arguments against? Is it about the fact that the output isn’t “optimal”?
DanielHB 2 hours ago [-]
Yes, but I also think it will be trivially by-passable if you pass your output through another LLM. At least for text.
It might help catch students cheating, but not real spam-bot usage. As soon as platforms start checking for watermarks spambots will add anti-watermark passes.
doc_ick 2 hours ago [-]
I read through their earlier announcement, i don’t think it’ll be trivially by-passable without distorting the original message.
I would agree it may not help spam-bot usage, however at this case seemingly the only user detection is likely an id/badge check, which is not good.
capten 2 hours ago [-]
Imagine telling someone they're wrong without providing any evidence or context.
Edit: I asked Opus 5 for "a jpg of a cat" via the web chat. It did it by writing an svg and then rendering that to a jpeg. Clicking the "download" button saved a jpg which did have C2PA metadata, seemingly injected by the artifact-presentation thing.
Then I told it to put it in a zip file. I downloaded the zip, and the extracted jpeg had no C2PA metadata.
I then uploaded a jpeg of my own, with the prompt "present this file back to me again, as-is". The downloaded file had C2PA metadata injected.
Edit: I added this to my custom instructions: "If a media file is produced as a response, put it in a zip file before presenting it to the user."
[1] https://www.anthropic.com/news/claude-text-watermark
[2] https://digital-strategy.ec.europa.eu/en/policies/code-pract...
[3] https://www.kqed.org/news/12095398/new-california-law-requir...
[4] https://www.leginfo.legislature.ca.gov/faces/billTextClient....
C2PA is file metadata and can be trivially stripped away, unlike hidden watermarks, e.g. SynthID.
For the text from an LLM, that you verify to be as true as if you wrote it yourself.
Not clear if making watermarks removable is a may, shall, or should, according to legislation. It certainly is not a must.
This is just a C2PA metadata checker.
That would invalidate the hash on minor changes. Too much effort and not enough return.
You can resave the file and the "made with Claude" signal disappears, but you cannot make a random file pass as Claude-made without Anthropic's signing key. So the useful guarantee is one-way. No signature means almost nothing.
At work, right after an AI training, we were asked to use our "authentic" voice when writing mid year reviews.
And there's tons of Claude-generated code there.
Also, I'd guess this is not just to prevent any AI-generated code in the training data, but specifically their own.
Percentage of users who put out their code on the web and also have a plan where Anthropic promises not to train on their data is problem also low.
So not excluding own code could be a real issue, since it would be impossible to deduplicate the training and RILHF data from their sessions with the code accessible elsewhere, and written by the very same users.
I'm wondering why they have restricted file types. You can't check a PDF for example... surely the main use case for people will be to check if a document was produced or edited by an LLM? That could be an attractive (if not misunderstood) proposition for academics
So I’m unsure what your aiming at.
Say I write a text by hand And then I tell it to clean up the grammar and fix some sentences Did it make it?
This is also interesting for those companies that siphoned the entire open web
That’s a cute way to imply their service is used to generate misinformation. They are basically saying to not trust the AI content made from their own product :)
Detection will get gamed, and a lot of useful text is already mixed. The more useful check, for me, is still human: would I put my name on this before it goes out?
Any attempts they use are defeated by a text editor and CTRL SHIFT V. Unicode characters are no new thing.
Reminds me of how people tried to argue that NFTs aren't anything more than just jpegs.
It might help catch students cheating, but not real spam-bot usage. As soon as platforms start checking for watermarks spambots will add anti-watermark passes.
I would agree it may not help spam-bot usage, however at this case seemingly the only user detection is likely an id/badge check, which is not good.