How do you serve Markdown to agents without breaking HTML?

A Markdown version of a help page can be useful to a client that wants plain text. The difficult part is keeping that version accurate and making sure a cache never sends it to a visitor expecting HTML.

Two workable designs are an explicit URL such as /docs/setup.md, or one URL that selects a representation from the request's Accept header. Choose the simpler design your clients can use. Neither design guarantees that an assistant will request it.

Decide who needs the alternative

Before building a second representation, inspect a real integration request. Does the client accept Markdown? Can it follow a link to a .md page? Does it already read the HTML correctly? Record the answer so the change has an identifiable consumer.

Markdown has a registered media type, text/markdown, documented in RFC 7763. That registration identifies a format. It does not establish an agent discovery convention or require AI products to prefer it.

For a small docs site, an explicit Markdown route is easy to explain and test. Its URL identifies the representation without a request header. Link to it from the HTML documentation index if it is intended for public use. Keep the ordinary HTML address available for people arriving from search or shared links.

If you negotiate, honor the request

HTTP content negotiation lets a client list acceptable media types, with quality values that express relative preferences. A robust implementation needs to interpret those values rather than search for the substring markdown. For example, text/markdown;q=0 does not express a preference for Markdown. MDN's content negotiation guide.

These are illustrative responses for a route that supports both formats:

Content-Type: text/html; charset=utf-8
Vary: Accept

Content-Type: text/markdown; charset=utf-8
Vary: Accept

Keep the response type truthful. Markdown text inside an HTML error page is not a Markdown response. Define what the route does when the client sends no preference, a wildcard or an unsupported format. Use a maintained negotiation library where possible instead of extending a handwritten header parser.

Check the cache in both directions

Vary describes which request headers influenced a response. For a negotiated route, Vary: Accept lets a compatible cache distinguish representations. MDN also says the value should be consistent across a URL's responses, including the default response and 304 responses. Vary reference.

Treat that as a configuration to verify at the public edge. Your CDN may have its own cache rules. A header emitted by the application is not proof that every layer uses it as intended.

Use a repeatable sequence:

1. Request HTML through the public hostname and save headers and body.
2. Request Markdown at the same URL and save both again.
3. Repeat HTML after the Markdown response has passed through the cache.
4. Repeat the experiment starting with Markdown.

For example:

curl -sS -D /tmp/html.headers -H 'Accept: text/html' \
  https://example.com/docs/setup -o /tmp/setup.html
curl -sS -D /tmp/md.headers -H 'Accept: text/markdown' \
  https://example.com/docs/setup -o /tmp/setup.md

These requests observe your format selection. They do not reproduce a verified crawler identity or a browser's complete behavior.

Publish the same facts in both versions

Generate the representations from one content source when practical. Compare required steps, warnings, limits and links. Tables need particular attention: a conversion that loses column labels may leave a price disconnected from its plan.

Test relative links from the actual Markdown URL. A link that works from /docs/setup may resolve differently from /docs/setup.md or a download host. Check code fences and headings as well as the text. Do not put credentials or private examples into a public alternate route.

If the Markdown converter fails, decide whether the route returns a clear error or falls back to a supported representation. Never present an empty successful document as a usable alternative.

Review the result with the intended client

Ask the client to extract a specific setup requirement from both versions and compare the answers with the source document. Save the responses and selected media types. That establishes whether this particular alternative helps your integration.

A Dedica scan can provide public website readiness findings as another input. Pair those findings with the representation and cache checks above; a score does not certify content parity or every client's format preference.

Sources

• The text/markdown media type, RFC 7763.
• Content negotiation, MDN.
• Vary response header, MDN.