004Field Note

FEATURED_INTELLIGENCE
6 min read·

AI Training Preferences Need a Provider-by-Provider Plan

Plan shared AI training preferences around each provider’s product scope, current controls and announced rollout dates.

#Multi-engine visibility#AI training preferences#Content governance
Share

CONTENT USE AND DISCOVERY

One preference, separate product decisions

review by provider
Shared vocabulary

Declare the training preference

  • Named __usage_ai-training group
  • Individual draft, not a final standard
  • Existing crawler rules still matter
Product-specific planning

Keep the scope and timing visible

  • Check training and grounding scope
  • Preserve current effective controls
  • Review each announced rollout
Documentation snapshot inspected November 18, 2026. No crawl, compliance or citation experiment performed.

AI training preferences now have a proposed shared vocabulary, but a multi-engine visibility plan still needs provider-specific decisions. The new __usage_ai-training token expresses a preference about using content for generative-model training. It does not settle every question about fetching a page, grounding an answer, or appearing in search. Before replacing existing controls, establish which provider, product, use and implementation date each decision covers.

That distinction matters for a publisher with a mixed goal: keep public product documentation discoverable while declining some downstream uses. A single company-wide instruction to “block AI” leaves the implementation team guessing. An equally broad instruction to “stay visible everywhere” leaves the content owner’s preference unresolved. The useful starting point is a purpose-specific decision that can survive a conversation between marketing, engineering and the person responsible for content policy.

What the shared signal actually proposes

The November 16 version of the Machine-Readable Training Preference Signal draft places a policy token in an ordinary robots.txt group. For example:

User-Agent: __usage_ai-training
Disallow: /

Within the draft’s model, this expresses a preference against the training use for the site. It is not a new crawler identity to find in access logs. The draft separates usage preferences from existing crawler rules, and a consumer may use a preference when deciding whether to fetch content.

Two parser details deserve attention: the draft excludes wildcard groups from matching this policy token, and a file without the named token expresses no preference through this mechanism. Neither observation means that other existing rules disappear. The document also discusses precedence for more specific proprietary policies.

This is an individual Internet-Draft, not an approved IETF standard or proof of universal implementation. Version -01 was published on November 16, 2026. Treat it as work in progress when describing it to stakeholders.

What the provider documentation says today

The following is a documentation snapshot inspected on November 18, 2026. Dates below describe announced plans, not a test of provider compliance on a real site.

OpenAI: its October 30 announcement supports the common training-preference signal and targets implementation in the coming months. It explicitly preserves existing crawler controls. That gives an operator a reason to prepare, but not a reason to delete today’s rules on the assumption that a rollout announcement replaced them.

Google: its common-crawler documentation states an intention to support the shared signal in early 2027. The same documentation describes Google-Extended as a control token with no separate request user-agent string. Its current scope includes future Gemini-model training and grounding in specified Gemini and Vertex AI products. Google says Google-Extended does not affect inclusion or ranking in Google Search. A product-specific control therefore cannot be described accurately as a switch for every Google AI or Search experience.

Microsoft: its training and grounding announcement says the new preferences begin taking effect from February 1, 2027 as Microsoft crawls. It separates training preferences from page-level grounding preferences, including a distinction between Microsoft-operated grounding experiences and grounding services for third parties. Its page-level training setting can override the robots.txt training setting. Existing NoArchive or NoCache users also have a documented transition process to review.

These statements are useful precisely because they are different. The common training vocabulary reduces one kind of duplication; it does not make the providers’ product boundaries and rollout plans identical.

Give the owner a choice they can actually approve

Consider a fictional software publisher with public setup guides and a small collection of licensed research reports. The team wants prospects to discover its setup guides. The research editor wants to decide separately whether reports can be used for generative training. Neither request is a configuration string yet.

Start with two plain-language questions for each content collection: which downstream use is acceptable, and which discovery experience should remain eligible? Do not ask the editor to approve a universal technical policy before the product scopes are understood. Conversely, do not let a marketing objective silently decide the research collection’s training preference.

For the setup guides, the owner might approve search discovery and some grounding while declining training. Engineering then checks whether each provider’s currently documented controls can express that combination. Where a provider couples uses within one control, the team has a tradeoff to resolve rather than a syntax error to fix. Record that unresolved choice explicitly; do not invent an equivalent setting.

For the reports, first establish whether they belong on the public web at all. Robots.txt is not a confidentiality mechanism. RFC 9309 says the protocol is not access authorization and warns that listed paths are publicly discoverable. A report that requires restricted access needs an appropriate access boundary. A preference about later use is a separate decision.

The example does not prescribe this policy for every publisher. Its purpose is to reveal where a broad “AI opt-out” request contains several different owner decisions.

Build a migration plan around the existing configuration

Before changing anything, save the served robots.txt and relevant page-level directives, identify who manages them, and list the content collections affected. This is an inventory, not an opportunity to clean up unrelated rules. A CMS, CDN and repository can otherwise each appear to own the same setting.

Then write a small provider-specific plan. Each entry should name the content collection, intended use decision, affected product scope, current control and documented implementation timing. Include the exact documentation URL and inspection date. Leave a field for an unresolved product-policy tradeoff instead of forcing every row into “allowed” or “blocked.”

Keep announced future support visibly separate from behavior the team has verified. Schedule a documentation review around the provider’s stated rollout rather than treating an early configuration change as proof that the provider has begun honoring it. For Microsoft sites already using legacy controls, read the actual transition instructions before editing; a generic shared-token example is not a complete migration guide.

After an approved change, inspect the file or response actually served for the intended paths. Check that other crawler-specific and page-level rules were preserved. A successful deployment proves the configuration was delivered. Provider behavior, search eligibility and any later business outcome still require their own evidence.

Keep the visibility diagnosis specific

For teams using GeoCompanion to turn an AI-visibility diagnosis into work, the useful issue is narrow: which content collection, product surface and use does this setting affect? “AI access fixed” hides the decision that the content owner needs to inspect.

Our earlier crawler-controls overview explains why access controls belong in a visibility discussion. This new shared preference adds a different planning task: maintain the distinction between a common usage vocabulary and each provider’s effective product controls during a transition. It does not establish that a particular page was crawled or cited.

A sensible handoff ends with an approved use decision, an owned implementation task and a dated review of the relevant provider documentation. It can also end with a specific unresolved tradeoff that goes back to the owner. Either is more useful than promising that two lines will preserve discovery, stop every unwanted use, or improve citations. None of those outcomes was measured for this article.

// AI_VISIBILITY_AUDIT

See how AI sees your brand

See your AI visibility across your site, content, and competitive signal, with the next fixes and priorities mapped for you.

Boost Visibility with AIAlready have an account? Sign in
// CREATOR_MOMENTUM

Need the creator-side next step?

Build your creator momentum on Launchvibes while GeoCompanion stays focused on AI visibility, content structure, and citation readiness.

Build your creator momentum

Join the GeoCompanion.ai Community

Connect with founders and marketers building stronger AI visibility, content systems, and next-generation execution.

Join Telegram
SIGNAL_PROPAGATION

Found this intelligence helpful? Propagate the signal across your nodes.