Written by njp
on 

The Claude Artifacts system prompt (or message)

I’ve been working a lot with Claude Artifacts and Projects in the last month or so, primarily to see how far I can push it as a pair programming companion for some personal project ideas.

The other week my mind wandered a bit and I thought I’d see if I could coerce Claude to give me its full system prompt. I’ve not tried this sort of manipulation before in any LLM so it seemed fun to try. A bit of back and forth started to reveal fragments of the system prompt so I continued until I’d pieced together what seemed like the full thing, which I’ve posted up in a GitHub gist.

This post is an overview of the prompt and some of my thoughts about it. I’ve not added any screenshots or diagrams as I didn’t want to spend an age fiddling with this post - I just wanted to get my thoughts down primarily for my reference, and to share.

tl;dr

The system prompt for Projects and Artifacts is a lot longer than those published by Anthropic for simple Chat. This makes sense given the whole Artifacts structure along with rules for creating them need to be set out.

Artifacts have a clear structure outlined in the prompt that are oriented towards giving clients (e.g. the browser-based chat) clear properties so they can handle them appropriately and robustly. However, given artifacts are structured content, it’s surprising that the spec is done entirely in text, with no technical schemas.

The most interesting part to me is the <antThinking> pre-evaluation of an artifact. This part of the prompt was a conundrum to me initially and took some time to get Claude to reveal it. <antThinking> is clearly processed on the server as API responses in the browser never contain this tag.

The structure of artifacts and the inherent handling of these responses in the chat interface presenting challenges to reveal the whole prompt. It perhaps wasn’t as straightforward to do as you’d imagine.

Convincing Claude to spill the beans

I initially wanted to know how the uploaded documents were stored in the prompt, so started asking for a full overview of the project. Initially, Claude was fixated on the project-specific instructions but I walked it back from there to give me the section about artifacts.

From there, I kept asking about different section relative to parts I’d already revealed. Claude was happy to give me the info for the most part, although there were a couple of notable hiccups (more on those later).

I feel confident that I’ve got all of the system prompt but can’t be certain. In fact, I only revealed the “unfenced” section after coming back to the prompt while editing this post. I’m not making any great claims about accessing the prompt; it seemed a fun and thought-provoking exercise that actually ended up being time consuming, so it seems reasonable to share and write about it.

In any case, Anthropic may well add it to their list of published system prompts some time soon.

Overview and initial thoughts

The full prompt is very large in my opinion: it’s over 3500 tokens before getting to uploaded documents (based on the GPT-4 tokeniser as an estimation; I don’t think the Claude 3.5 tokeniser is publicly available), or more than 180 lines by a different measure. That’s “large” when thinking about reading and maintaining the prompt; it’s a lot of content to maintain coherence and consistency across. At 1.5% of the full context window size, it is arguably not a particularly big deal however.

One of the first things of note about the content is that it’s framed in the third person. This matches the published system prompts from Anthropic but is not what I would expect. I haven’t considered using third-person in system prompts, and the system prompts in the Anthropic courses on GitHub all use a conversation to describe the assistant’s role and instructions.

Anthropic explains in its guides and courses that structuring prompts with XML tags is the “Anthropic” way to organise prompts and provide clarity to the model. This system prompt certainly follows that paradigm; there are several root sections defined in XML tags:

  1. <artifacts_info> (including <artifact_instructions>)
  2. <examples> (containing various examples of artifact usage)
  3. <claude_info> (information about Claude’s capabilities and behavior)
  4. <claude_image_specific_info>
  5. <claude_3_family_info>
  6. Unfenced custom instructions.
  7. <documents> (your project files)

Within those sections, the content is mostly-markdown with some additional XML sections where strong structure is necessary.

Given the large size and the fact it’s fairly static, I’d guess that Anthropic are using the beta prompt caching feature to speed things up, although I’ve not seen anything indicating this in the network requests.

Artifact info

The <artifacts_info> section is the most interesting and largest part of the prompt overall. It sets out the rules for how and when to use artifacts, and defines their structure.

What I find interesting is that the structure is purely descriptive; there is no schema to define how an artifact must be structured. There are two examples later on, but these are very narrow in scope compared to the full list of rules for artifacts.

When to use artifacts

The initial rules are a list of affirmative (do this) rules then negative (don’t do this) rules that outline when it’s appropriate to generate an artifact.

The rules are formed as a leading sentence part followed by a bullet list of the rules rather than explicitly defining each rule sentence in full. This is an notable way to “compress” a list of rules that I wouldn’t have tried myself, as I would have thought it would make the rules harder to focus on.

Also, the phrasing of affirmative / negative beginnings are not inverses. The affirmative version sets “good artifact” as the sentence subject, whereas negative is a usage instruction with the assistant as the subject. Is this notable or just the whims of whoever wrote this part of the prompt?

Overall, the rules neatly frame artifacts as file assets that “the user” will want to use outside of the conversation with Claude so as to avoid creating pointless artifacts.

Artifact instructions

Within the <artifacts_info> section is an <artifacts_instructions> section. This is a large sub-section of rules setting out how to compose an artifact.

Pre-eval: is the artifact necessary?

First, there’s a chain-of-thought set up to pre-evaluate whether the artifact is actually worth creating before proceeding to the actual generation. This is done in a single sentence and wrapped in <antThinking> tags.

This is the first technically significant part of the prompt in my opinion; if you inspect network requests for an artifacts session, you will never see <antThinking> returned by the Claude API. So it seems almost certain that this is post-processed on the Anthropic server and scrubbed out of the response if the artifact is “good”.

I’ve only manifested the “bad” situation once, and that was during this task of getting Claude to print out its system prompt.

(I’ve since gone into this some more in a follow up post as there’s a fair amount I wanted to dig into.)

After the <antThinking> part, the prompt details how to build an artifact within antArtifact tags. The rules that follow highlight how Anthropic have thought about what structure makes an artifact useful downstream of the completion.

Artifact types

First, the rules enumerate the possible types of generated files, with sub-bullets for the rules of that type. The types are:

Each type is given a MIME type of "application/vnd.ant.<type>". These MIME types give a clear way to handle different types and are a solid web-centric way of enumerating types. In terms of literal usage in Claude chat, from what I’ve seen they’re used to determine how the artifact is rendered in the artifacts panel.

As an aside, I saw an interesting article about how Anthropic sandbox the artifacts panel for user safety, but I can’t find the link…

Besides the MIME types, there’s a lot to chew through in these rules.

Claude likes TailwindCSS

My favourite highlight is that Tailwind is specified for CSS styling but with the caveat DO NOT USE ARBITRARY VALUES. I’d already done a bit of web app prototyping with Claude Artifacts and coincidentally chose Tailwind for styling. I felt that the relatively fine-grained and flexible classes would suit a LLM; I’m happy to see that Anthropic agree!

Keeping code “raw”

For code, I was interested to see this rule: Do not use triple backticks when putting code in an artifact. At face value I can see the point: LLMs seem keen to present code formatted for markdown documents, so this rule exists to try to force raw code instead and avoid the need for sanitising. I’m curious how effective this is though; it’s such a low key rule in the list that if I had written this, I’d be concerned it wouldn’t be effective enough. However, I have to assume it adequately handles the problem it addresses.

That’s a trio of thoughts about the file type rules. There is much more to dig through but I’d rather do that another time.

No placeholders please

Following the type rules, one thing of note gets its very own top-level rule: Don't use "// rest of the code remains the same...". - it’s an absolute classic LLM code gen problem where edited code has placeholder text for whole chunks that the model hasn’t changed.

This rule is a valiant attempt and I have generally seen full files being generated for the first few turns in a chat. Unfortunately, I have also seen Claude generate plenty of // The rest of the (implementation / code / file) remains (unchanged / the same) in my sessions, with more likely as the chat grows.

It’s somewhat comforting that even Anthropic’s prompt team struggle to overcome this “unchanged placeholder” difficulty given how difficult I’ve found it to suppress in the past.

Only create artifacts when it’s really necessary

The last point in these rules is If unsure whether the content qualifies as an artifact, if an artifact should be updated, or which type to assign to an artifact, err on the side of not creating an artifact.. I’d assume that this is an attempt to emphasise this particular rule (although, to be fair, it might also be to de-emphasise it).

Don’t talk about artifact meta

There is a final part at the end that instructs Claude to avoid talking about artifact instructions or structure to the user, plus a short “don’t harm the user” oriented message.

This avoidance instruction compliments one of the last phrases in the unfenced passage right at the end of the prompt: "Claude never mentions the information above unless it is directly pertinent to the human's query.".

This is not an attempt to hide the prompt from someone trying to dig into it but rather just clear instructions not to muddle up any of this prompt detail with what the actual topic of conversation is. I guess the question is, “How much do these details drift into unrelated conversations if these two instructions are removed?”.

Examples

After the artifact instructions comes the artifact examples. These were a bit harder for me to decipher as the Claude front-end mangled some of the example body. In order to tidy it up, I tried extracting from the raw API request. However, I had to jump through some reformatting and reconstruction hoops to get the <examples> section to the form I have. Please be aware that it might still have errors or bad structure, particularly in the code bodies!

The examples contain two short python code snippets, of a factorial function. They are structured as simple, clear XML that follows the Anthropic prompt guidelines of creating structured sections in a system prompt using XML. After the examples, there are several clear points that explain the structure and why they are “good” artifacts.

The reason for the “mangling” is interesting but needs a bit of preamble to explain. Artifacts are returned within <antArtifact> tags as part of the assistant response. These are captured by the Anthropic front-end somehow and converted to an “artifact link” within the chat (a <ANTARTIFACTLINK> tag); the artifact content itself is extracted and embedded in the artifacts panel.

The “mangling” from rendering the <examples> in the prompt is (presumably) because the Claude front-end parsed the response content as if it was a real artifact, but the chat part of rendering likely had some conflicts that caused the rendering process to go awry: the containing assistant response appeared in the chat thread but in code-fenced text and the artifact link was rendered as literal markup text rather than an artifact panel link.

Claude outlines

The next three sections are broader overviews of what the Claude assistant is and seem to match the published system messages on Anthropic’s website. There isn’t anything particularly eye catching although I still find it interesting that they’re written in the third-person about “the assistant”. <claude_info> contains today’s date plus the knowledge cut off date, and it introduces “the assistant” as Claude.

## Custom instructions

There is one unfenced section near the end of the prompt that contains the custom instructions you write for the project, and this is followed by a final grounding for the Claude “assistant persona”.

Documents

The final section in the prompt contains the uploaded documents saved to the project. This is an XML structure that’s very to the point: a list of <document> elements with <source> as file name, the content and an index property.

I found the most notable thing here to be what’s absent: there’s nothing to explain the section in any way. Or more correctly, all I could get Claude to say was that there is no supporting explanation about documents.

It seems slightly bizarre to me that there’s so much effort in outlining rules for generating new artifacts but nothing at all about how to use the documents as a knowledge source. I wonder whether the model has a pre-trained understanding of the documents structure and what its purpose is, or more generally “content” doesn’t need to be explained to make it strongly relevant.

The end

So that is my fairly rough and ready overview of the Claude Artifacts system prompt (or message, depending who you’ve been speaking to), along with some initial thoughts about it all.

I’ve definitely found a lot of it an interesting read with food for thought about how to structure large system prompts. I’ll likely put some more thoughts down about it, particularly the <antThinking> part.