Yes, a new common signal in robots.txt should allow site owners to refuse to use their content to train generative models without automatically blocking traditional search engines. But it is not necessary to install it as a magic formula: the text is still an Internet-Draft, the calendars differ according to OpenAI, Google and Microsoft, and the grounding of IA responses remains a separate use whose processing varies according to the provider.
On 30 October 2026, a technical project defined the use ai-training token. Placed in a robot.txt group, it allows to express a preference at the scale of a site or certain paths. OpenAI announces that it will apply in the coming months, Google in 2027 and Microsoft from 1 February 2027.
This convergence is useful for creators, media, trainers and independents who publish original content. It reduces the need to write a different rule for each company. However, it does not regulate past usages, robots that refuse to cooperate, or the protection of confidential content. Above all, it does not mean everywhere the same for IA responses powered by the Web.
Information verified on 31 October 2026. The status of the project and the implementation dates may still change.
What was actually announced on October 30 2026
The main technical source isInternet-Draft Machine-Readable Training Preferences Signal. He proposes to use the already known syntax of robots.txt with a token of common policy:
User-Agent: use ai-training Disallow: / Allow: /press-kit/In this example, the site expresses a preference against generative training for all its contents, except those located under /press-kit/. The mechanism does not create a new directive: it reuses User-Agent, Disallow, Allow and robot priority rules.txt.
The project defines IA training as the use of content to modify the parameters learned from a generic model. It excludes systems used exclusively for classification, classification or rating. If no group use ai-training exists, no preference is expressed under this mechanism. The generic User-Agent: * group should not be interpreted as automatically covering this token.
We must also present our status with precision. The IETF sheet indicates that it is an individual Internet Draft, not approved by the IETF and without formal standard status. This first version is dated from 30 October 2026 and expires on 3 May 2027 if it is not updated, replaced or extended. It is therefore an ongoing work, even if large suppliers already announce that they want to adopt it.
OpenAI plans to respect the signal in the coming months and publish details of its implementation later. Its current controls remain valid. Google announces support in 2027, while its documentation on common robots evokes the beginning of 2027. Microsoft Bing gives a more precise date: the 1 February 2027, with a transition to be completed before 1 March for publishers still using NoArchive or NoCache.
Training, research and grounding: three uses not to be confused
The main point is the purpose. Training changes the parameters of a future model in a sustainable way. Research explores and indexes pages to make them appear in results. Grounding, or anchoring, uses content at the time of a request to help an AI produce a more current response and possibly cite its source.
The new token aims to generate training. The draft states that it is not a crawl directive: the rules for exploration robots continue to apply separately. Therefore, it should not be inferred that a Disallow for use ai-training automatically prohibits classical indexing.
At OpenAI, the separation is already legible in the robot documentation. GPTBot refers to content that can be used for training, while OAI-SearchBot is used to make sites appear in ChatGPT's search functions. A publisher can currently block the first and authorize the second. The common signal must simplify the drive preference, without immediately replacing these OpenAI controls.
Microsoft announces explicit separation. The training preference can be expressed in robots.txt or in one page. The grounding will have separate page settings: Allow, DisallowExtended to limit usage to experiences operated by Microsoft, or Disallow. Microsoft states that these choices do not alter eligibility for traditional Bing results, but that a refusal to grounding greatly reduces visibility in the responses generated.
The Google case requires more caution. Google indicates that, if the common signal refuses training and no Google-Extended rule is explicitly defined, it provides to interpret it as a Google-Extended refusal. However, according to google documentationGoogle-Extended covers the training of future generations of Gemini but also certain uses of grounding. The common signal remains a training preference; Its operational translation at Google can nevertheless affect eligibility for Gemini grounding. Therefore, before any changes are made, the two groups should be compared and the current documentation checked.
This distinction extends the VIFLY guide on the visibility in ChatGPT and IA agents. Being quoted by an answer, appearing in research and contributing to future training are not three ways of saying the same thing. These are three different editorial decisions.
Two concrete examples for professionals VIFLY
A trainer protects her premium supports while leaving her expertise visible
A trainer publishes free articles, a presentation page and course excerpts. It also hosts some very detailed public resources under /training/. Its aim is to stay discoverable in Google, Bing and IA engines, but to refuse that publicly accessible course materials are used for general training.
She starts by mapping her paths. Discovery articles may remain without common preference or receive an explicit Allow if they wish. The media directory can receive a Disallow. It shall maintain supplier-specific controls in parallel until the common signal is effectively applied. For content reserved for customers, it does not use robots.txt as protection: it retains authentication and real access permissions.
This choice allows him to reason by value and purpose. Its pages intended for discovery remain open to the routes she considers useful; his working resources receive a stronger preference. It documents the decision so that it can be revised when implementations are stabilized.
A photographer wants to be quoted without automatically yielding his images to training
An independent photographer wants his reports to appear in the search results and can be cited when a user is looking for a local professional. On the other hand, he does not want to express an agreement to train generative models from his portfolios.
It does not blindly block all IA robots. It distinguishes between research robots, training robots and user triggered accesses. It also checks the real weight of the original images: a public miniature does not have the same role as a high definition file delivered to a customer. The originals remain behind a controlled access and are never protected only by a public robot line.txt.
This approach joins the VIFLY analysis on the links and profiles in Google AI Mode : useful visibility depends on public content, attributable to and linked to a sustainable source. Refusal to use training does not mean abandoning any discovery strategy.
The VIFLY point of view: control usages, not visibility out of fear
For a creator or independent, publishing performs several functions. A page can establish expertise, attract research, reassure a prospect, sell a resource or serve a client. Treating all these pages with a single global rule often confuses different assets.
The new signal makes a question that already existed visible: what counterpart do you expect from each public content? A reference article can gain from staying discoverable and citable. A paid model, a full course, a photographic base or a client document should not be made public simply to facilitate referencing. The first protective measure then remains the access architecture, not the robot file.
The common signal can become a useful governance brick because it reason by use rather than by robot brand. But the simplicity of the syntax must not mask the complexity of the decision. Google, Microsoft and OpenAI still do not align all the consequences on grounding, IA responses or historical controls.
Article VIFLY on meta and Crawl conditions of sites already recalled that a robot rule.txt is a signal to automated customers, not a security barrier or a universal legal response. The new token improves the expression of a preference; He doesn't turn a voluntary protocol into a safe.
Nuances, limitations and points to watch
The text is not yet an adopted IETF standard. It may be modified, replaced or abandoned. The announced implementations can also adjust their calendar or interpretation.
The signal is not yet applied everywhere. OpenAI talks about the coming months, Google's 2027 and Microsoft's 1 February 2027. Publishing the rule today expresses an intention, but does not prove that a supplier has already detected or complied with it.
The perimeter is global, but without uniform country warranty. This is not a feature reserved for the United States: a French or Reunion site can publish the syntax. On the other hand, the consequences depend on each company, its products and its activation date. No separate list for France or Reunion is announced.
The rule works for future collections. It does not erase the content already collected, does not impose the withdrawal of an existing model and does not replace a suitable legal or contractual request.
Non-cooperative robots can ignore it. The protocol is based on consumer interpretation and respect for the signal. It does not technically block a download.
Grounding remains separate. Microsoft provides separate settings. OpenAI maintains differentiated robots. Google foresees a particular interaction with Google-Extended. Check the supplier before announcing that a training refusal will always preserve each form of AI visibility.
robots. txt is not a security. The RFC 9309 specifies that the listed paths become public and discoverable. A client space, a paid resource or a confidential file must use authentication and access controls.
The European context does not create an automatic response. The Code of good practice for general-purpose d recognizes reserves of machine-readable rights and cites robots.txt. This reinforces the practical interest of the signal, but the qualification of rights, the application of copyright and possible remedies require a situation-specific analysis. The folder VIFLY AI helps distinguish law, platform rules and good practices without confusing them.
What to do now? An eight-step action plan
- Invent your public content. Separate discovery pages, articles, portfolios, free resources, extracts, premium content and customer areas.
- Decide by purpose. For each group, indicate whether you accept classical research, appearance in IA responses, grounding and future training.
- Check the current robots.txt. Find GPTBot, OAI-SearchBot, Google-Extended, ClaudeBot, Bingbot and generic rules. A new rule can come into tension with an older choice.
- Keep supplier-specific controls. The common signal is not yet everywhere active. Do not delete an existing rule only because the project has been published.
- Test the perimeter by road. Start with a controlled environment or a group of clearly identified pages. Check the Allow and Disallow priorities before targeting the entire domain.
- Really protect what must remain private. Use connection, permissions, temporary URLs and storage suitable for paid, client or confidential files.
- Document the decision. Note the date, objective, relevant paths and additional checks. This trail avoids a future technical change inadvertently cancels your choice.
- Return to the 2027 settings. Check the final implementation of OpenAI, Google support and Microsoft transition. Also check server logs, indexing and actual visibility.
Therefore, the priority is not to copy a line immediately. It is to build an understandable policy: what content should be found, which can feed a response, which should not lead to a model and which should simply not be public.
Frequently Asked Questions and Sources
What code does it allow to refuse IA training for an entire site?
The project proposes a User-Agent group: use ai-training followed by Disallow: /. It is still an Internet Draft and its respect depends on the participating suppliers.
Does this signal block Google Search or Bing?
No, not as such. The project separates the use preference for training from the Crawl rules for engines. Google and Microsoft state that preference does not affect eligibility for traditional research, subject to normal rules for exploration and indexing.
Can we refuse training while remaining visible in IA responses?
Sometimes, but the setting depends on the supplier. Microsoft separates training and grounding. OpenAI distinguishes its research and training robots. Google plans an interaction with Google-Extended which requires a specific verification.
Is the signal already respected by OpenAI, Google and Microsoft?
Not yet generally at 31 October 2026. OpenAI aims for the coming months, Google 2027 and Microsoft on 1 February 2027. Existing controls must remain in place during the transition.
Does the rule protect confidential files?
No. robots.txt is public and is not an access control. Sensitive or paid files must be protected by authentication and authorization.
Does the signal apply in France and in La Réunion?
The syntax can be published by any site. No geographical restrictions are indicated in the project. However, its real effect depends on the timing and implementation of each supplier.
Sources
- IETF Datatracker — Machine-Readable Training Preferences Signal, 30 October 2026.
- OpenAI — Making it easy for websites to express AI training preferences, 30 October 2026.
- Google Search Central — Supporting a shared AI training control for websites, 30 October 2026.
- Google — Common crawlers and Google-Extended, consulted on 31 October 2026.
- Microsoft Bing — Your content, your choice, 30 October 2026.
- OpenAI — Overview of OpenAI Crawlers, consulted on 31 October 2026.
- RFC 9309 - Robots Exclusion Protocol.
- EU AI Act — General-Purpose AI Code of Practice, measure 1.3.