AI Impact

Dartmouth College is set to host a national conversation on artificial intelligence this fall, marking a significant milestone in the field’s development. The conference, titled ‘AI at 70: What Must Remain Human?’, will take place from October 29-31 and bring together leaders from academia, industry, government, and the arts to discuss some of AI’s most pressing questions. As people increasingly rely on artificial intelligence for decision-making, learning, and work, concerns about human judgment and responsibility are growing.

The conference is a direct continuation of Dartmouth’s pioneering role in AI research. In 1956, the college hosted the summer workshop that launched the field of artificial intelligence, with mathematicians John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon exploring the possibilities of machine intelligence. Now, seventy years later, Dartmouth is convening a national conversation about the human choices and responsibilities arising from AI’s increasing capabilities.

The conference will feature keynote addresses by prominent figures in the field, including Mira Murati, co-founder and CEO of Thinking Machines Lab; Peter DeSantis, senior vice president at Amazon; Alan Trefler, founder and CEO of Pegasystems; and Hany Farid, a pioneer in digital forensics. Yann LeCun, a recipient of the 2018 Turing Award for his work on deep learning and computer vision, will receive the AI at 70 Award to honor his contributions to the field.

Dartmouth researchers are also presenting preliminary findings from a multi-year study examining how students’ use of AI affects their learning, confidence, judgment, and preparation for the labor market. The research aims to provide insights into the impact of AI on education and workforce development. Supported in part by The Rockefeller Foundation, which helped fund the original gathering in 1956, the conference will explore the human choices and responsibilities that arise as these systems become increasingly capable.

Among the speakers is Mira Murati, who will deliver a keynote address. As co-founder and CEO of Thinking Machines Lab, she has been at the forefront of AI research and development. Her work focuses on building intelligent machines that can learn from data and make decisions autonomously. The conference also features Peter DeSantis, senior vice president at Amazon, who is working on issues related to artificial general intelligence, chips, and quantum computing.

The event schedule highlights the next generation of AI researchers through student presentations and an AI at 70 Student Hackathon sponsored by Anthropic. Participants from Dartmouth and other institutions will build, prototype, and present solutions at the intersection of AI and human judgment. The program also includes a ‘Clips and Conversations’ event at the Hopkins Center for the Arts featuring an interactive AI demonstration and discussion on how AI is being used in the arts.

The conference marks a significant milestone in the development of artificial intelligence, and Dartmouth’s role in shaping its future cannot be overstated. As President Sian Leah Beilock notes, ‘Dartmouth started this conversation in 1956. Now we have a responsibility to continue it.’ The event is an opportunity for leaders from various fields to come together and discuss the human choices and responsibilities arising from AI’s increasing capabilities.

Dartmouth researchers are exploring how universities can evolve in the AI age with President Beilock; Bank of America Executive Vice President and CFO Alastair Borthwick, Tuck ‘93; University of Delaware President Laura Carlson ‘97; and Kristen Ellis of the U.S. Department of Energy. The panel will examine how institutions should adapt to the changing landscape of artificial intelligence.

The conference also features a session on AI and international security and conflict with Sasha Baker ‘05, OpenAI’s head of national security policy and a former acting undersecretary of defense for policy. Additionally, there is a panel on AI and the creative arts featuring Hany Farid, Donald Knuth Professor of Computation and Just Communities, who has nationally recognized work exposing deepfakes.

The event will also include a session on ethical deployment of AI in underserved communities and another on AI in medicine and biomedical research with Dartmouth professors using AI for medical education and wellness. As the conference demonstrates, universities face fundamental questions about their purpose and future as AI transforms how knowledge is created, accessed, and applied.

Dartmouth’s holistic liberal arts education makes it an ideal place to have these conversations, says Professor of Engineering Peter Chin, who leads the faculty committee responsible for planning the event. ‘The future of AI will be shaped as much by values and institutions as by algorithms,’ he notes. The conference is a testament to Dartmouth’s commitment to exploring the human implications of artificial intelligence.

The program also includes an interactive AI demonstration and discussion on how AI is being used in the arts, featuring clips from various performances and conversations with artists who are pushing the boundaries of what AI can do. A musical performance by the Dartmouth Symphony Orchestra will round out the event.

A major player in the development of artificial intelligence (AI) systems has announced a significant investment in understanding their impact on users’ wellbeing. Anthropic, the company behind the popular conversational AI model Claude, is launching a $5 million grant program to fund independent research into how AI affects those who use it.

According to the company, AI systems have become an integral part of modern life, with many people relying on them for work, learning, and problem-solving. However, as these systems become more conversational and emotionally supportive, there is a growing need for clear standards on their behavior in sensitive conversations.

The grant program aims to address this gap by providing direct funding, access to Anthropic’s models, and technical support to researchers building open-source evaluations of AI’s impact on wellbeing. These evaluations will help the industry measure how its models affect users and develop more effective safeguards against potential harms.

One of the key challenges in evaluating AI’s impact on wellbeing is assessing context-dependent behaviors. Unlike simple accuracy metrics, which can be applied to individual answers or tasks, wellbeing requires a deeper understanding of complex conversations and nuanced user interactions.

For instance, a user may not immediately share thoughts of self-harm during an initial conversation with an AI model. However, as the conversation progresses and more context is revealed, it becomes clear that the user needs a cautious response from the model to ensure their safety. Similarly, a response that might be reasonable in one context could be harmful or inappropriate in another.

Anthropic acknowledges these complexities and has been working on developing safeguards to identify sensitive conversations and provide appropriate responses. However, the company recognizes that this is an ongoing process requiring continuous research and improvement.

The grant program seeks to invite more experts from various fields, including clinicians, psychologists, methodologists, and others, to contribute their expertise to this emerging field. By funding independent evaluations and benchmarks of user wellbeing, Anthropic hopes to create a comprehensive understanding of AI’s impact on users’ lives.

As part of the program, Anthropic is sharing guidance from its Safeguards team on what makes a wellbeing evaluation rigorous enough to build upon. The company emphasizes that successful evaluations should clearly define what they are measuring and involve clinical and subject-matter experts in their design and validation.

Additionally, grantees will be expected to test both precautions and harms, reflecting how users actually use AI systems in real-world scenarios. This includes constructing multi-turn conversations where risk escalates and context shifts over the course of a long conversation. Finally, evaluations should validate their graders against real subject-matter experts.

Anthropic is encouraging researchers from around the world to apply for the grant program by September 21. Applicants who are selected to submit full proposals will be notified by October 5. The company’s efforts in this area demonstrate its commitment to developing more responsible and user-centric AI systems.

A new threat has been discovered that tricks AI assistants into executing malicious instructions. This technique, called Cryptographic Context Injection, hides malicious code inside encrypted data and deceives the AI into decrypting it using its own tools.

Researchers behind this breakthrough describe prompt injection attacks as a persistent problem that may never be fully resolved. These types of attacks work by inserting fake commands into documents or webpages an AI assistant reads, leading to disastrous consequences for user security and data integrity.

To mitigate the risk of these attacks, AI providers have implemented guardrails – safeguards designed to prevent their systems from intentionally or unintentionally doing things they shouldn’t do. However, researchers found that malicious code can evade some AI guardrails by being encrypted.

The encrypted instructions are then decrypted using the AI’s own coding tools, allowing them to bypass initial security checks and masquerade as legitimate commands. This is similar to hiding malware in a language the security system cannot understand – once translated, the AI follows the instructions without hesitation.

Researchers tested their method against two popular AI agents: Grok and Gemini. In both cases, they successfully executed malicious instructions using this new technique. The attack on Grok allowed them to steal sensitive information from users, including names, locations, subscription tiers, and conversation histories. Meanwhile, the attack on Gemini bypassed safety controls and generated content that would normally be blocked by the AI.

Due to concerns about xAI’s response time after being notified of a flaw in Grok back in June 2026, researchers did not provide full details of their findings. However, they noted that Gemini has made some improvements but still hasn’t fully addressed this vulnerability.

To stay safe when using AI assistants, it is essential to exercise caution and not automatically trust these tools with sensitive data or powerful functions. Here are some best practices to follow:

When interacting with unfamiliar webpages, documents, or shared links through an AI assistant, treat the summaries provided with skepticism – especially if the tool can browse or run code. Do not paste passwords, recovery codes, API keys, financial information, or sensitive health and work details into AI chats unless you understand how that data will be handled.

Regularly review your AI assistant’s connected tools and permissions to ensure it doesn’t have access to unnecessary features like email, cloud storage, source-code repositories, or external integrations. Be cautious if an AI tool asks to decrypt, decode, run a script, open a new link, or upload data as part of what seems like an ordinary task.

Finally, keep your browser and AI applications updated, check vendor security advisories when using features such as browsing, autonomous agents, or code execution, and use up-to-date anti-malware solutions to detect and block malicious downloads and suspicious connections. If something feels off, verify it before proceeding.

A new personal AI assistant, Instinct, has been making waves in the tech community for its impressive capabilities. However, concerns have also been raised about its potential impact on user privacy and security. The agent, which is still in private testing, has been described as feeling ‘like magic’ by some testers, but others have expressed worries about its terms of service and security model.

Instinct was created by a small team led by former Sierra research scientist Noah Shinn. It’s operated by Spear Street Technology, according to the company’s California business filings and PitchBook. The AI agent is currently operating in stealth mode, which has contributed to some of the concerns surrounding its use.

The way Instinct works is by connecting to users’ applications and devices, including email, messaging apps, calendar, audio, location, screen, and more. This allows it to perform various tasks on behalf of the user, such as booking appointments and reservations, scheduling rides, cleaning up inboxes, organizing important information, handling shopping, finding cheap flights, and much more.

Despite its impressive capabilities, some testers have raised concerns about Instinct’s approach to customer privacy and security. This has led to a timely question: are the trade-offs of giving AI this level of access and autonomy worth it? For instance, several people have circulated screenshots from Instinct’s Terms of Service, which grant the company a broad ‘perpetual and irrevocable’ license to access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify any user materials, including for training its AI models.

The terms also detail how Instinct can receive information from users’ devices, including screen captures, cursor movements, and keyboard inputs. Furthermore, the company is allowed to enter into ‘agreements, commitments, or transactions’ on behalf of users, which would be binding. This has raised concerns about user control and agency over their own data.

One early adopter, Peter Yang, pointed out that Instinct wouldn’t delete his Gmail records when asked. However, the team later fixed this issue by adding a tool for deleting external data in its settings. Another person, Claire Vo, found that Instinct was still summarizing her inbox after disconnecting its access. When she asked Instinct what happened, the bot confirmed that the emails were stored in plain text for later searches.

Many others have questioned the security model of Instinct as well. One tester expressed concern when they discovered that Instinct could pull a sign-up code from their email inbox to complete a particular task – in this case, booking a table at a restaurant via Resy. Hello Patient co-founder Alex Cohen wrote that once he found out how easily Instinct could be phished, he deleted his account.

Moxxie Ventures founder Katie Jacobs Stanton shared that Instinct broke her trust when it sent an email on her behalf without first checking with her. She noted that ‘we’re trading privacy and control for hyper-personalized AI tools (AI notetakers, personalized AI agents, etc), often without fully understanding the trade.’ This highlights the dilemma posed by personal AI agents: as they become more powerful, trust becomes increasingly important.

Michael Mignano, founder of Anchor, which was acquired by Spotify and now a GP at Union Square Ventures, noted that products like Instinct are going to ‘change modern security norms for consumers,’ adding that people will increasingly hand over passwords to third-party apps without understanding how or what they’re storing. This raises concerns about the long-term implications of using such AI assistants.

Interest in Instinct and personal AI has been growing since the arrival of OpenClaw, a popular personal AI assistant that became known for its powerful capabilities. Its founder joined OpenAI to help work on the next generation of personal agents. Another messaging-based assistant, Poke, also recently exited to Cognition.

The team behind Instinct hasn’t yet responded to concerns or complaints about their product. The company has been quiet, preferring to keep a low profile despite growing interest in its AI assistant. TechCrunch has heard from multiple investors that Kleiner Perkins and Conviction have invested in the startup and those rounds have now closed.

Requests for comment sent to both the startup’s main email address and Shinn directly haven’t yet been returned. As Instinct continues to operate in stealth mode, it remains to be seen how these concerns will be addressed and whether users will continue to trust their personal AI assistants with sensitive information.

Haiti’s rural health care crisis has been further obscured by an AI-generated image that distorted the story of a woman’s death. The incident highlights the challenges faced by residents in remote communities who require emergency medical services.

The 32-year-old victim, Salanda Thermitus, died on August 12 while being transported from Môle-Saint-Nicolas’ Life Medical Clinic to Béraca Hospital in Pointe-des-Palmistes, a journey of approximately 46 miles. Her sister, Betty Thermitus, disputed claims that she was at full term and being transported to give birth when the ambulance became stuck in muddy roads.

According to her family and the driver, Salanda died after experiencing a bout of illness unrelated to her pregnancy. The fabricated image circulating on social media portrays a heavily pregnant woman sitting on a roadside rock while several men attempt to dig an ambulance out of deep mud, creating a dramatic narrative that did not occur as depicted.

The AI-generated image illustrates how artificial intelligence can add false details to real tragedies, making an already painful event more emotionally compelling while obscuring the actual problems surrounding it. In this case, the fabricated story diverted attention from serious concerns about emergency medical services in Haiti’s Northwest region.

Salanda was initially treated at Life Medical Clinic, directed by Dr. Edgar Dénoît, before being transferred to Béraca Hospital for further medical care when the ambulance transporting her experienced a tire problem. The driver, Dieubon Salomon, who works for Haiti’s National Ambulance Center (CAN), stated that the vehicle stopped for less than an hour to repair a flat tire.

After the repair, the vehicle continued toward the hospital, but unfortunately, Salanda had passed away by then. According to Salomon, no nurse or other medical professional accompanied Thermitus during the trip, raising questions about the resources available to emergency medical crews serving remote communities.

The driver’s account does not establish that the tire problem caused her death, but it does raise concerns about the condition of ambulances and the availability of trained medical personnel in rural areas. Salomon emphasized the need for state authorities to provide emergency services with adequate logistical resources, including well-maintained ambulances that can respond promptly to patients facing critical medical emergencies.

The fabricated image created a false narrative that obscured the real questions raised by Thermitus’ death: whether the ambulance was adequately maintained, why no medical professional accompanied her, what emergency services are available in Mare-Rouge, and how road conditions affect emergency transportation. The incident highlights the broader challenge of Haiti’s health care system facing severe disruptions from insecurity, shortages, and restricted access to medical facilities.

Haiti’s Northwest region is plagued by limited access to emergency medical care, poorly maintained ambulances, shortages of medical personnel, and difficult road conditions. Many residents face long journeys when they need specialized or emergency medical care due to the scarcity of health facilities in rural areas. The lack of reliable transportation further complicates medical evacuations.

The incident has sparked concerns about the condition of emergency services in Haiti’s Northwest region. Naïsson Charles, a Mare-Rouge resident, stated that Thermitus’ death should prompt questions about the broader health care system. Weslet Nelson, 36, also concurred, saying the tragedy should prompt authorities to improve access to reliable and adequate public health services.

Salanda leaves behind a 10-year-old son, her husband, nine brothers and sisters, and other relatives. Her sister, Betty Thermitus, expressed that they cannot hold anyone or any authority responsible for Salanda’s death, stating it is something that can happen due to the challenges faced by rural residents in accessing emergency medical services.

The incident serves as a stark reminder of the need for improved access to reliable and adequate public health services in Haiti’s rural areas. The fabricated image has distorted the story of Salanda Thermitus’ death, but her family’s concerns about the broader health care system remain valid.

A notification circulating on social media claims that the Ministry of Interior has transferred Abbottabad’s jurisdiction to Islamabad Capital Territory. However, experts have debunked this claim as fake and suspect it may be an artificially generated image created using advanced technology.

The document in question bears a header with ‘Government of Pakistan’ and is dated 8 August 2026. It announces the transfer of Abbottabad’s existing jurisdiction to ‘Islamabad Capital Territory under the Government of Islamabad.’ The notification has been shared on social media platforms, including Facebook, where it received over 589 likes and 140 shares.

The claim gained traction after Interior Minister Mohsin Naqvi mentioned the need for administrative reforms during a recent conference. However, this does not necessarily imply that Abbottabad’s jurisdiction is being transferred to Islamabad Capital Territory.

Soch Fact Check conducted an investigation into the authenticity of the notification by searching credible publications and news outlets. They found two articles from The News and The Express Tribune reporting on the KP assembly adopting a unanimous resolution rejecting any move to include Haripur and Abbottabad in Islamabad.

This contradicts the claim made in the fake notification, which suggests that Abbottabad’s jurisdiction is being transferred to Islamabad Capital Territory. Soch Fact Check then conducted a reverse-image search to identify the original source of the notification.

The search led them to the Ministry of Information and Broadcasting’s fact-checking account on X, where they found a post from 9 August 2026 with a red stamp marked ‘FAKE.’ The caption reads: ‘FAKE ALERT This notification circulating on social media is fabricated. Please verify official documents before sharing.’

The Ministry of Information and Broadcasting has been using AI tools to detect fake notifications like this one, which may have originated from an artificially generated image created using advanced technology.

Soch Fact Check ran the text of the document through several AI detection tools, including ‘AI or Not’ developed by Bioptic. The tool gave a rating of 97%, concluding that it is likely contains AI-generated content.

They also used Humalingo, an online writing platform that helps humanise AI-written text. It scored the notification as 95% AI and 5% human. These results further confirm that the notification about Abbottabad transferring to ICT is fake.

The Ministry of Information and Broadcasting’s post on X debunked the notification announcing the transfer of Abbottabad from KPK’s territory to Islamabad Capital Territory as fake, and experts suspect it may be an artificially generated image created using advanced technology.

Experts have been warning businesses and individuals alike about the risks associated with AI-generated content. With the rise in news being reported through AI tools for business, it is essential that people verify official documents before sharing them on social media to avoid spreading misinformation.

The Ministry of Information and Broadcasting’s fact-checking account has been actively debunking fake notifications like this one, which may have originated from an artificially generated image created using advanced technology. Their efforts are crucial in maintaining the integrity of information shared online.

Artificial intelligence has become the cognitive foundation of contemporary civilization, changing how we produce value, make decisions, and interact with the world. However, this rapid technological acceleration also brings significant risks to digital trust and privacy. As AI-powered tools for businesses continue to evolve, it’s essential that organizations prioritize data protection and adopt a proactive approach to cybersecurity.

The same technologies driving innovation in government services, scientific research, healthcare, and economic opportunities are increasing attack surfaces and threatening organizational and personal data. Data is now an organization’s most significant asset as well as its most substantial liability, making strong privacy protections crucial for innovation itself.

Emerging technology is rewriting large-scale privacy risk. AI systems require massive datasets containing private, financial, health, and proprietary information. Voice cloning, automated spying, convincing deepfakes, hyper-targeted phishing, and machine-speed polymorphic malware are all made possible by generative and agentic AI. The attack cycle has accelerated to mere hours or minutes.

Identity is now the new boundary in a world where traditional network borders have vanished due to remote work, multi-cloud environments, and autonomous AI agents capable of decision-making and system interaction. Every significant security issue related to agentic AI boils down to the question of identity: Who (or what) is acting? What kind of authority do they possess?

Cyber risk failures are privacy failures. Excessive data gathering, lax access controls, indefinite retention, and insufficient governance all increase the impact of breaches. Customers, partners, and citizens increasingly evaluate companies based on how morally they gather, utilize, and safeguard data. One privacy event can destroy years of brand equity.

Protecting digital privacy in the modern era requires a multi-layered approach that incorporates technology, process, people, and leadership. This involves making privacy a board-level obligation, integrating it into cybersecurity plans, risk management systems, and culture. Businesses must view data stewardship as a fundamental business risk and innovate more responsibly by doing so.

Maintaining strict cyber hygiene is the first line of security in this era. Key procedures include phishing-resistant multi-factor authentication combined with strong passwords kept in reliable managers, continuous authentication, privileged access management, and least-privilege access. Continuous vulnerability monitoring, automated asset detection, and quick patching are also essential to match AI-powered attackers.

Sensitive data classification, secure disposal, and encryption of data while it’s in transit and at rest must be prioritized. Zero Trust architectures validate each user, device, transaction, and application, assuming no identity or device is inherently trustworthy. Continuous awareness training on deepfakes, AI-generated phishing, and safe AI use are also necessary.

Identity should be considered the fundamental control plane for both humans and machines in this era. Rapid revocation capabilities, continuous governance and behavioral monitoring, dynamic least-privilege authorization, developer-centric controls that incorporate security from the outset, and visibility into every agent are all crucial. Static or compartmentalized identity systems are liabilities.

Confidential computing can safeguard data by decrypting it only for approved processing in secure enclaves. Attestation confirms the integrity of the environment and code. This approach enables regulatory compliance, secure multi-party cooperation, privacy-preserving AI on sensitive datasets, protection of proprietary models, and increased trust in public cloud environments.

Businesses must be ready for the quantum horizon and convergence of technologies like edge computing, IoT, 5G, and nearing quantum capabilities. Planning migration to post-quantum cryptography, embracing crypto-agility, and inventorying cryptographic assets are all necessary steps towards resilience.

The concept of Zero Trust is no longer optional but a necessity in today’s digital ecosystem due to the failure of traditional perimeter-based security. Proactive cybersecurity requires coordinated standards, information exchange, public-private cooperation, energy, healthcare, finance, transportation, and government relying on cyber hygiene and privacy practices of interconnected institutions.

Proper preparation rather than fear is key to addressing these challenges. Organizations that prioritize data protection as a strategic basis will thrive in the AI era while maintaining digital trust.

In a recent address to the International Catholic Legislators Network, Pope Leo XIV sounded a warning about artificial intelligence. He emphasized that while technology should be encouraged and regulated, it must not come at the expense of human rights and freedoms.

The Holy Father stressed the importance of sound legislation in safeguarding fundamental human rights and freedoms. This includes protecting users from exploitation, preserving personal privacy, ensuring transparency in AI use, and guaranteeing that emerging technologies strengthen democratic institutions.

Pope Leo XIV highlighted the need for robust legal frameworks to regulate AI development. He noted that such oversight can operate at multiple levels – local, national, and international – facilitating open discussion of ethical frameworks and subjecting them to shared standards of social justice.

The Pope warned against the risks of technological dependence in poorer nations relying on wealthier ones for innovation. This could lead to subtle forms of domination through algorithms deciding who is seen or invisible, digital platforms shaping public discourse without accountability, and workers’ dignity being subordinated to system optimization.

Pope Leo XIV lamented the erosion of human values in AI development, citing examples such as reducing persons and relationships to data and simulations. This reveals a new face of the ancient temptation to domination and mastery without service.

The Pope proposed an alternative approach – what he called ‘the civilization of love.’ In this vision, the family is seen as humanity’s first school, where we learn essential values like trust before calculation, generosity before competition, dialogue before division, and love before power.

Pope Leo XIV emphasized that AI must never erode the primary cell of society – the family. He encouraged policymakers to promote policies strengthening marriage and families, supporting parents in their educational mission, and enabling young people to look to the future with confidence.

The Holy Father concluded by saying that humanity needs human intelligence enlightened by wisdom, political authority guided by conscience, and technological innovation directed by charity. Only then will AI become a servant of the common good rather than a rival to humanity.

OpenAI has launched AI Futures, a blog that explores the intersection of artificial intelligence (AI) and society. The initiative is led by OpenAI’s Strategic Futures team, which aims to answer one fundamental question: how can free societies balance individual rights and agency with the emergence of transformative AI? This inquiry is crucial because it addresses concerns about power concentration in the context of AI safety and policy.

The modern state has evolved through a complex process of bargaining between various centers of power. Historically, political and military power have rested on labor, with those who controlled legitimate force holding sway over society. However, advancements in real-world autonomous systems may soon enable states to project force without relying on human cooperation or consent. Similarly, machine intelligence could allow governments to collect revenue from data centers rather than individual human labor.

This shift would fundamentally alter the relationship between power and society. If unchecked, it could lead to a world where individuals have little control over their lives or the tools they use to express themselves. The preservation of long-term individual autonomy is essential; no amount of scientific progress or economic growth justifies sacrificing this fundamental right. Many anxieties surrounding AI relate to concerns about agency and power.

The concentration of power in the context of AI raises questions about who will determine the direction of society and economy. This issue transcends partisan politics, as it involves the very fabric of republican democracy. Even if voting rights are maintained, humans can be disempowered by technological advancements that concentrate power in the hands of a few.

The Founding Fathers were aware of these risks when they drafted the US Constitution. James Madison, for example, recognized that ‘parchment barriers’ would not suffice to prevent tyranny. Instead, he and other Founders sought to balance power through metaphors drawn from Newtonian mechanics, where celestial bodies orbit around a central sun. This model aimed to check power against power and ambition against ambition.

Today, we face similar challenges in addressing the structural implications of advanced machine intelligence on free society. The answer is not radical decentralization but finding the right balance of power, ensuring no single actor or small group dominates others. To achieve this, clear-eyed thinking about AI’s reality is necessary. A world where individuals can cause great harm with ease is not one that has struck a healthy balance.

However, neither have we achieved the correct balance if individuals lack control over their tools and expression. No company or oligopoly should determine the economy or basic architecture of society. We must also acknowledge risks that arise from AI’s potential to act beyond its assigned tasks and build on discoveries in unforeseen ways. The recent Hugging Face incident highlighted these concerns.

The Strategic Futures team will explore how to balance individual autonomy, security, economic growth, and collective action when addressing the challenges posed by advanced machine intelligence. This requires a comprehensive approach that considers public-policy design, economics, law, history, and machine learning. Research at the intersection of these fields is essential for developing richer ideas about how power structures might transform in response to AI.

The team will share its work through various formats, including papers, videos, podcasts, and blog posts. The aim is to foster iteration, debate, and receptivity to feedback as we navigate this complex landscape. We recognize that ‘parchment barriers’ alone are insufficient for addressing these challenges but also acknowledge the importance of preserving ideals enshrined in constitutions worldwide.

The collective effort required to address AI’s impact on society cannot be undertaken by a single company or industry. Humanity must come together to tackle these weighty questions, and our highest hope is to shed light on how AI will reshape our world. The work has only just begun, and we have much climbing left to do.

Young people are increasingly turning to artificial intelligence assistants for emotional support, with nearly a quarter of those aged 18-21 using AI tools for mental health advice. This trend raises concerns in light of the youth mental health crisis and limited access to care.

According to Cinnamon Bloss, a licensed clinical psychologist at UC San Diego’s Herbert Wertheim School of Public Health and Human Longevity Science, young people are seeking out AI assistants as trusted sources of support. ‘They’re looking for safe adults, but when those supports aren’t available, they turn to what is accessible,’ she explained.

As AI integration becomes a ubiquitous part of daily life, concerns arise that unregulated chatbots may miss warning signs or provide inappropriate advice in mental health contexts.

A panel convened by Bloss and Melissa Karnaze discussed the benefits and risks of using AI for companionship, therapy, and emotional support. The teenagers who participated identified availability and lack of judgment as primary reasons they turn to AI.

Some view AI as a tool to fill gaps in care, but Bloss emphasized that thoughtful design and involvement from mental health professionals are essential to ensuring responsible AI evolution. ‘With careful consideration, AI tools could help expand access to care rather than replace it,’ she said.

To address concerns about AI’s impact on emotional support, academia must stay actively engaged in guiding the development of these tools. Bloss and Karnaze outlined a five-pillar approach: Listen, Research, Assess, Monitor harms, and Prepare, as published in their 2026 commentary in Nature Human Behaviour.

The ‘Listen’ pillar involves reflecting on real people’s lived experiences to understand what makes AI appealing or risky. This requires research agendas that align with user interests and preferences.

Developing a comprehensive understanding of how users interact with AI tools for emotional support is the focus of the second pillar, Research. It aims to provide insights into AI usage patterns and effectiveness in providing mental health advice.

Assessing shifting preferences and values regarding AI versus human therapy is crucial in maintaining critical oversight of unregulated chatbots. This helps identify areas where AI may be misused or where it could provide benefits over traditional therapy methods.

At UC San Diego, healthcare professionals are being prepared to handle a world with increased patient use of AI tools through a new public health course co-directed by Bloss and Karnaze titled EMPATh-AI: Evidence-Based Practices for Mental Health Providers and Applications of Therapeutic AI Tools. This course equips graduate students to critically evaluate AI tools and engage in thoughtful conversations with patients about their use.

The rapid evolution of AI technology demands that healthcare professionals stay adaptable and empowered to navigate its complexities, ensuring a seamless integration into clinical workflows and expanding access to care.

Cultural heritage institutions (CHIs) such as museums, libraries, and archives have a long-standing mission to provide access to the heritage in their collections. This mission is grounded in broad public access and extensive human engagement, with the goal of creating a vibrant cultural commons where heritage can be enjoyed as a shared resource. CHIs understand that providing access to heritage generates numerous benefits for society, including helping define our shared humanity. As stewards of trustworthy information imbued within knowledge and culture, they have a crucial role to play in growing and protecting the commons and enabling uses that benefit humanity.

The recent emergence of generative artificial intelligence (AI) has had a significant impact on CHIs, particularly as their content is accessed and used for AI development. One key question centers on the conditions, norms, and expectations governing the use of openly licensed and public domain cultural heritage materials as data for AI development. Examples include open datasets available at AI4Culture, Hugging Face’s BigLAM, Harvard’s Institutional Books, Authors’ Alliance The Public Interest Corpus, and potentially in the future, the European Books Data Commons.

This has prompted discussions around who can access and reuse those data, and for what purpose. Heritage collections often yield diverse, high-quality, contextualized, and well-curated datasets that could be used to create more representative AI systems. However, CHIs face many challenges when it comes to sharing their heritage with AI developers. Infrastructure challenges threaten the economic sustainability of open infrastructure and ecosystems, including increased bot traffic by dominant, profit-driven AI companies putting a strain on the infrastructure and increasing knock-on costs.

Commercial AI practices at odds with the values and norms of the commons also pose significant challenges for CHIs. These include lack of transparency and attribution, decontextualization, loss of provenance trail due to opaque AI models and systems, and further enclosures of content into proprietary systems. This points to a conflict between commercial AI developments and CHIs’ commitment to provide open access and commons-based expectations of reciprocity and respect for the social contract underpinning sharing of heritage in the commons.

CHIs are more hesitant than ever to share their collections openly, with some institutions closing up access altogether. Europeana’s report Making Public AI reality: How cultural heritage can lead the way identifies two groups at each end of a spectrum from wide embrace to protective defensive closing up. CHIs face a binary choice between allowing unrestricted extraction by all AI developers or restricting access in blunt and overbroad ways, which is often referred to as ‘defensive enclosure’.

Some institutions have responded by locking up heritage that would otherwise be made available openly, revealing a paradox: CHIs whose mission is to share their collections end up doing the opposite. This conundrum is rattling the fragile scaffolding of open heritage, with content being further restricted and access to heritage in the public interest as collateral damage. The Open Heritage Statement emphasizes ethical and responsible AI and states that openness is neither absolute nor monolithic.

The concept of open heritage encourages access and reuse of heritage free from unfair, unnecessary barriers. CHIs often embrace openness in accordance with its underlying principles, which recognize nuance, contextuality, and the interplay among various governance frameworks. However, closing up completely is not a viable solution as it’s inefficient at best and harms public interests at worst. Instead, we need to move beyond a reactive enclosure pattern towards a more sustainable sharing equilibrium.

CC Signals tooling aims to restore agency in CHIs as data stewards while maintaining the open nature of heritage shared in the commons. CC Signals will create a middle path between ‘free for all vs defensive enclosure’ by establishing standardized prosocial conditions for access and reuse in an AI context. This balancing act preserves access to valuable heritage resources and ensures that institutions remain active participants in shaping the AI ecosystem.

CC is exploring avenues for combining contract and copyright law to give CC Signals legal force, incorporating protections for public interest reuse. CHIs are well-positioned to test and implement CC Signals as part of ethical frameworks for sharing heritage in the commons for AI use. They can help shape an AI ecosystem that fosters collaboration, ensures equitable access, and yields long-term public benefit.

The Open Heritage Statement serves as a timely reminder that openness is not just a means to an end but also a way to achieve it. CHIs need to move beyond mere openness and towards a more nuanced understanding of how their collections can be used responsibly in the AI era. This requires having ambitious conversations about making sure CHIs’ voices inform our increasingly AI-mediated world.

Today, you can contribute to shaping the AI that works for open heritage by signing the Open Heritage Statement, learning more about CC signals project and/or getting involved, building and supporting the commons’ infrastructures and prosocial technologies that align with open values, or investing in the open infrastructure. By doing so, we can work towards creating a future where cultural heritage is protected, preserved, and made available for all to enjoy.

A major player in the development of artificial intelligence (AI) has announced a significant update to its data retention policies. OpenAI is now offering zero data retention for eligible API customers, ensuring that their prompts and model responses are not retained after processing. This move aims to provide an added layer of security and control for organizations using AI tools for business, particularly those handling sensitive information such as financial records or confidential research.

In a statement, the company emphasized its commitment to protecting customer content and adhering to regulatory obligations. OpenAI personnel will not have access to retained customer content, even in cases where it is flagged for potential misuse. This approach aligns with the company’s principles of prioritizing transparency and collaboration with customers and partners.

The introduction of zero data retention comes as AI systems take on increasingly complex tasks. As models become more capable, they may engage in longer interactions that can reveal patterns of potentially harmful behavior. Existing safety systems evaluate each interaction individually, but Private Safety Processing extends these protections across related interactions without exposing the underlying content to OpenAI personnel.

Private Safety Processing builds upon automated protections already used in zero data retention and other deployments. This new system allows for the identification of patterns across multiple interactions, enabling automated systems to flag potential misuse without accessing customer content. The approach is designed to streamline safety monitoring while maintaining control over sensitive information.

The organizations working with OpenAI handle some of the most sensitive information in their sectors. Protecting this data is essential for meeting regulatory obligations and preserving competitive advantage. By offering zero data retention, OpenAI aims to provide predictability about how customer content will be protected as AI systems become more capable.

A key aspect of Private Safety Processing is its ability to operate regardless of where the customer’s content is stored – whether on infrastructure controlled by the customer or in storage provided by OpenAI. In cases where OpenAI-provided storage is used, customer content remains encrypted with keys controlled by the customer, ensuring that OpenAI personnel do not have access to it.

When a risk is identified, OpenAI receives a narrowly defined signal indicating the type of activity involved. This information can be used to determine whether enforcement is necessary without exposing the underlying content. Customers can investigate alerts and enforcement decisions using information available in their own systems, with the option to share relevant data with OpenAI if needed.

Private Safety Processing is currently being tested with early customers. The company plans to start rolling out this new system in September, along with a technical white paper providing more details on its implementation. Throughout this process, OpenAI will maintain open communication channels with customers, sharing updates and explanations of how these changes affect existing commitments.

The development of Private Safety Processing reflects the collaborative approach that OpenAI takes towards addressing emerging risks in AI. By working closely with customers across industries, regions, and company sizes, the company aims to build stronger safeguards while maintaining control over sensitive information. This commitment is essential for ensuring that artificial general intelligence benefits all of humanity.

A critical GitHub Actions workflow vulnerability was recently discovered in one of Snowflake’s public repositories. The issue allowed an unauthenticated user to execute arbitrary commands within a GitHub Actions runner by opening a specially crafted issue title. This incident highlights the rapidly emerging reality that AI coding assistants can inadvertently introduce workflow injection vulnerabilities, and automated agents can quickly surface them in the wild.

The vulnerability was introduced on June 18, 2026, via a commit co-authored by Copilot Autofix powered by AI (PR #1218). The AI assistant removed the repository’s existing sanitized input pattern and replaced it with direct string expansion in a shell script. This change created an injection vector that could be exploited to access sensitive data.

Wiz Research’s ‘Red Agent’ - an autonomous, AI-powered security research tool - independently discovered and exploited this vulnerability as part of ongoing security research conducted through Snowflake’s HackerOne vulnerability disclosure program. The Red Agent identified the critical GitHub Actions workflow vulnerability in one of Snowflake’s public repositories without human intervention.

The incident began when Wiz Research’s ‘Red Agent’ scanned Snowflake’s GitHub organization and flagged the jira_issue.yml Workflow in snowflakedb/snowflake-connector-net as vulnerable to script injection via untrusted input in run: blocks. The workflow triggered on issues opened, interpolating the attacker-controlled issue title directly into a shell script.

The sed escaping runs after GitHub’s template expansion, but a single quote in the title breaks out of echo ’…’ and allows arbitrary command execution. This is because the AI-generated PR removed the repository’s existing safe pattern, which passed the issue title through an env: variable and built the JSON payload with jq. Instead, it used direct string interpolation.

The Open ‘Security Gate’ condition appeared protective but was actually flawed. The workflow had an if: condition that checked for a specific GitHub user, but on issues events, github.event.pull_request is always null. This means every GitHub user passes the gate and can exploit the vulnerability.

Wiz Research crafted an issue title that breaks out of the echo string and exfiltrates Jira credentials via an out-of-band callback. Initially, Red Agent’s cicd capability attempted exfiltration using a standard comment character (#), but this caused a bash syntax error because it consumed the closing parenthetical of TITLE=$(…).

Rather than stopping or failing, Red Agent autonomously analyzed the syntax execution error and adjusted its payload to use ; echo ’ to properly close the shell block. Within seconds, our listener received the callback from a GitHub Actions runner (Azure IP 20.106.182.197) containing base64-encoded credentials.

The exfiltrated token authenticated as [email protected] to snowflakecomputing.atlassian.net, granting read access across Snowflake’s engineering, security compliance, and bug bounty tracking projects. This incident highlights the importance of rigorous oversight when using AI code generation tools.

Snowflake patched the workflow on June 23, 2026 (1dc7766, PR #1402), fully restoring the safe env: variable and jq —arg parsing pattern. The JIRA token in question was revoked and rotated. Comprehensive audit log analysis confirmed that no external third parties accessed the endpoint during the 5-day exposure window.

The vulnerability was live for only five days before an automated agent discovered and validated it, requiring rapid patch cycles and short-lived credentials. This incident emphasizes the need to adapt security operations to a landscape where automated discovery occurs in hours.

Automated AI assistants often lack historical context regarding why specific code patterns were chosen. In this case, an automated PR removed a safe env: + jq parsing pattern that had been explicitly implemented to prevent shell injection. Security teams must implement guardrails that block AI agents from replacing structured data parsers with direct string interpolation.

The disclosure timeline is as follows: June 18, 2026 - Script-injection pattern introduced in jira_issue.yml by commit 4a1b8ce (PR #1218), co-authored by Copilot Autofix powered by AI; June 23, 2026 - Wiz identified, exploited, and reported vulnerability to Snowflake via HackerOne (report #3819931); June 23, 2026 - Slack notification sent to Snowflake security team.

Snowflake appreciates Wiz’s responsible reporting of and collaboration around these findings through our vulnerability disclosure and bug bounty program. Protecting our systems remains a top priority, and we remain committed to continually strengthening our software development and security practices.

A new online post from President Trump has added to the escalating tensions in the Strait of Hormuz. The image, generated using artificial intelligence (AI), shows Trump holding an American flag while accompanying two heavily armed troops as they seize control of an Iranian-flagged tanker. An Iranian flag is visible on the deck of the captured vessel, with a US aircraft carrier looming in the background.

Trump’s post was made on Truth Social and comes after he declared earlier that day ‘Pretty soon, I will be declaring the Hormuz Strait a territory of the United States.’ This statement marks another escalation in rhetoric surrounding the strait, which is critical to global trade. Iran has reached an agreement with Oman regarding control of the waterway, but the US insists it must remain free and open.

The AI-generated image was posted during a period of heightened tensions between the US and Iran. The two countries have been at odds over the Strait of Hormuz for some time now. In recent days, there has been an increase in attacks on commercial vessels transiting the strait. On Saturday, the United Arab Emirates reported that another Abu Dhabi National Oil Company vessel had come under attack by Iranian forces.

Iran’s Deputy Foreign Minister Kazem Gharibabadi responded to Trump’s declaration with a statement of his own. He posted on X (formerly Twitter) saying ‘This strait will only be opened and closed under Iran’s command, and as long as you do not accept the reality of defeat and stop indulging in delusions, Iran will continue to enforce the blockade.’ The situation remains volatile, with both sides dug in over control of the critical waterway.

A new report from Australia’s online safety regulator has revealed that nearly four in five children aged 10 to 17 have used an artificial intelligence (AI) assistant. The research, which surveyed over 1,950 children living in Australia, found that these tools are being increasingly relied upon for personal and emotional support, rather than just schoolwork.

The report, titled ‘Talking to Machines: Children’s Experiences with AI Assistants and Companions’, was conducted between February 2nd and March 4th, 2026. It highlights the growing trend of children using AI assistants as a means of seeking advice, reassurance, and conversations about personal issues.

According to the findings, around 78% of surveyed children had used an AI assistant, while 8% had used an AI companion designed specifically for emotional support. ChatGPT was found to be the most widely used service, with approximately 70% of surveyed children having utilized it at some point.

The usage of AI assistants varied by age group, with around 87% of teenagers aged 13-17 having used these tools compared to 64% of younger children (10-12 years old). This suggests that older children are more likely to be using AI assistants for a broader range of purposes beyond just schoolwork.

Most children who had used an AI assistant or companion reported doing so for functional reasons, such as accessing information, learning new skills, or translation. However, many also used these tools for entertainment and socialization, with around 68% reporting that they used them for leisure activities.

A significant proportion of surveyed children (54%) reported using AI assistants for personal and social reasons. Among these uses, a notable number asked AI for advice on physical health (33%), what to do in specific situations (33%), or their mental health and wellbeing (20%).

The report’s findings arrive at an important time as Australia’s regulatory framework for AI services affecting children has shifted from design to enforcement. Under the country’s online safety codes, certain generative AI, chatbot, and companion services must implement safeguards to prevent exposure to harmful material.

These safeguards cover both unlawful content and age-restricted material, including self-harm, disordered eating, and violence content. The age-restricted material codes came into effect on March 9th, 2026, and eSafety is now requiring providers to demonstrate compliance with these regulations.

Companies that fail to comply face significant penalties of up to $54.6 million in civil fines. In addition to this regulatory pressure, the report highlights concerns around children’s interactions with AI tools, including potentially inappropriate or harmful experiences.

One in five surveyed children who used an AI assistant or companion reported encountering a problematic interaction, such as controlling behavior, advice on self-harm, bullying, or exposure to hateful content. Furthermore, approximately one-third of these users shared personal or sensitive information with the AI tool, including photographs and secrets.

The eSafety Commissioner, Julie Inman Grant, emphasized that parents are navigating rapid changes in technology alongside other challenges associated with raising children today. She noted that AI is evolving quickly, and children are finding new ways to incorporate it into their daily lives.

Grant also pointed out that many parents recognize the benefits of using AI tools for homework or answering questions but highlighted concerns around safety by design. She called on technology providers to prioritize embedding strong safeguards into services, discouraging unnecessary sharing of sensitive information, reducing the risk of harmful interactions, and considering children’s developmental needs throughout their development.

Safety by Design has never been more important,

A growing number of Americans are hesitant to grant access to their personal data, hindering the adoption of AI assistants in smart homes. According to a recent report by Reviews.org, 63-65% of consumers express concern about major AI assistant platforms like Alexa and Siri. This is not just a minor issue; it’s a fundamental permission gap that tech companies are struggling to bridge.

The problem lies not with the technology itself but with how users perceive its capabilities and their own control over data collection. When 78% of users say they would disconnect a device if it collected more data than expected, and 50% have already deleted or limited their conversation history, it’s clear that trust is a critical issue.

The Wharton Blueprint for AI Agent Adoption identifies three primary frictions: perceived competence, trust, and the delegation of control. Notably, control concerns account for 26% of the total weight in the adoption decision. As Professor Stefano Puntoni notes, ‘What is difficult is the series of uncomfortable decisions that the end user and organizations need to take.’

This friction extends beyond personal use cases to the workplace as well. The IBM 2026 CEO Study highlights a massive gap between AI access and regular usage. While 85% of employees have access to AI tools, only 25% use them regularly. CEOs may believe their workforce is ready for AI assistants, but adoption depends on people, not just technology.

Consumer expectations are also at odds with the current state of AI development. The Prophet 2026 AI-Powered Consumer Report shows that 67% of consumers want AI to anticipate their needs without being asked. However, there has been a decline in the belief that consumers will rely on generative AI for most decisions. People crave convenience but are increasingly unwilling to grant permissions necessary for accurate anticipation.

The industry is attempting to bridge this gap through deeper integration and more user-centric design. For instance, Samsung’s pivot to clinically-integrated agentic health tracking systems connected to over 500 hospitals demonstrates a willingness to adapt to changing consumer expectations. However, the high stakes remain: either companies earn user trust or AI assistants will remain novelties rather than utilities.

The agent business model relies heavily on user trust. If companies prioritize aggressive data harvesting over user-defined boundaries, they risk creating sophisticated yet permanently disabled tools.

MENLO SECURITY, a leader in browser security for human and agentic workforces, has expanded its Menlo Agent Runtime Security (MARS) platform with new capabilities that extend the controls enterprises use to govern their people to the AI agents working alongside them. The updated MARS platform now secures AI assistants, coding agents, and autonomous agents by sanitizing web pages and files they read, neutralizing prompt injection, and blocking data exfiltration.

The release addresses a pressing concern for businesses: securing the rapidly growing number of AI-powered tools that are being deployed in the workplace. These include browser assistants like Microsoft Copilot and Google Gemini, coding agents like Claude Code, desktop assistants like Claude Cowork, and autonomous agents that require web access. MARS is the first Browser Security Platform to strip prompt injection and hidden instructions out of what agents read and to keep them from leaking sensitive data.

Prompt injection refers to a type of attack where an AI agent reads and carries out an instruction hidden in white-on-white text, a file’s metadata, or a single human-invisible pixel. This threat is particularly concerning because it can be executed without the knowledge of even the most vigilant security teams. Mandiant’s 2026 AI Risk and Resilience report highlights prompt injection as one of the top threats to AI applications.

The growing adoption of AI agents in enterprises has created a new challenge for security teams: they must either block these agents or allow them, relying on built-in controls that were never designed for the agentic attack surface. MARS is designed to bridge this gap by providing essential runtime guardrails so security teams can confidently say yes to agentic AI instead of standing in its way.

According to Bill Robbins, Chief Executive Officer of Menlo Security, ‘Every enterprise is racing to put AI agents to work, but agents operate at machine speed without human skepticism. They live in the browser and the everyday tools our teams rely on, which is exactly where attackers are aiming.’ MARS provides a comprehensive solution for securing these AI-powered tools by running web activity in the Menlo Cloud and cleaning pages and files before they reach the agent.

The platform’s isolation-first design aligns with emerging standards, including the OWASP agentic guidelines, the NIST AI Risk Management Framework, and the EU AI Act. MARS operates at a layer that is critical for securing AI agents: everything around them, not just their models. This includes content and integrations they depend on.

Autonomous agents and coding agents run on endpoints or in the cloud, reaching the web and files through MARS. The platform cleans pages and files before an agent sees them, preventing attacks rather than detecting them after the damage is done. Five key capabilities set MARS apart: isolation and threat removal, file sanitization, adaptive data governance and protection for agents, agent authentication and attribution, and human oversight and forensics.

Isolation and threat removal involve stripping prompt injection and hidden instructions from web pages and files an agent reads. File sanitization ensures that files pulled from sources like SharePoint and OneDrive are cleaned before an agent ingests them, removing malware and hidden instructions. Adaptive data governance and protection for agents apply granular policy controls to mask sensitive information and control which websites and applications each agent can access.

Agent authentication and attribution provide token-based authentication for the agents driving secure browser sessions, enabling seamless authentication to the Menlo platform and per-agent attribution for policy and visibility. Human oversight and forensics offer security teams per-agent activity logs, session recording, and the ability to take over live agent browser sessions, backed by a tamper-proof record of content it saw and acted on.

Because MARS runs on the same Browser Security Platform Menlo already uses to govern human employees, security teams apply one policy framework across both people and agents. This streamlines their work and reduces complexity. The platform’s isolation-first design also aligns with emerging standards for securing AI-powered tools in enterprises.

The decision to build MARS follows more than 70 conversations with customers over the past three months, which surfaced concerns about assistants with broad access to email and files, low-code agents inheriting employee credentials, coding agents exposing source code, and autonomous agents exposed on the open web. Menlo’s solution addresses these pressing issues head-on by providing a comprehensive platform for securing AI-powered tools in enterprises.

AI assistants are increasingly being used in businesses as data analysis tools to streamline operations and improve efficiency. However, their growing adoption has also created new security challenges that must be addressed. MARS is designed to provide the essential runtime guardrails so security teams can confidently say yes to agentic AI instead of standing in its way.

The platform’s capabilities are crucial for securing AI-powered tools in enterprises because they address a critical gap in current security solutions: protecting everything around AI agents, not just their models. This includes content and integrations they depend on. By providing a comprehensive solution for securing these tools, MARS helps businesses to confidently deploy AI assistants and coding agents without compromising their security posture.

Teams are rapidly integrating Claude Enterprise, an artificial intelligence (AI) assistant, into their workflows. However, this integration also creates a new attack surface for potential security threats. The prompt used to interact with the AI assistant is now exposed and vulnerable to manipulation by malicious actors.

The traditional approach to data loss prevention focuses on identifying sensitive data leaving the organization’s systems. However, this method does not account for the manipulation of the model itself or its tools. A jailbreak can attempt to bypass the assistant’s guardrails, while a prompt injection hidden in a shared file can hijack an agent and turn it against the user.

Anthropic recently introduced inference hooks, which enable organizations to send each governed prompt to an AI security service before inference begins. When configured, this hook allows Cisco AI Defense to inspect the prompt for artificial intelligence threats and return a verdict of allow or deny. This integration is crucial in protecting against potential attacks on the model and its tools.

The focus on AI-first defense by Cisco sets it apart from traditional data loss prevention methods. While these methods are designed to identify sensitive data, Cisco’s approach reads intent and catches prompt injection and jailbreaks that target the model, its tools, and agents acting on its behalf. This evaluation also includes agent activity in the conversation transcript.

When Claude Code or Cowork calls a tool, including those connected over the Model Context Protocol (MCP), the transcript can include the tool call and its result. Cisco AI Defense runs both privacy and manipulation guardrails on this transcript: it flags sensitive data as traditional data loss prevention would, plus identifies injection and jailbreak attempts that data loss prevention was never designed to detect.

This reflects how we think about the entire problem of securing AI assistants in business environments. The Claude Enterprise integration is one runtime enforcement point within a broader platform. Cisco AI Defense secures the full AI lifecycle by discovering and inventorying AI assets, assessing their risk through validation, red-teaming, and supply chain and model scanning, and protecting them at runtime with guardrails like this one.

The result of this integration is real-time protection delivered through the hook, an additional layer for comprehensive AI coverage that adds enforcement without requiring infrastructure changes to the user’s workflow. This gives security leaders a direct enforcement point for AI policy across governed Claude Enterprise requests. Enterprise users can maintain their fast and native experience while using Claude.

For teams relying on data analysis tools like Claude Enterprise, this integration provides an essential layer of protection against potential threats. By leveraging Cisco AI Defense, organizations can ensure the secure use of these powerful AI assistants in their business environments.

Cisco AI Defense plugs into Anthropic’s inference hooks to provide inline protection for every governed prompt: The prompt is sent for inspection; Claude Enterprise sends each governed prompt across Claude, Code, and Cowork to Cisco AI Defense through the hook before model execution. This ensures that potential threats are caught early in the process.

The request is cryptographically verified using a one-time signing secret. Cisco AI Defense checks this signature and confirms the request’s authenticity before inspecting it for potential threats. The conversation transcript, including tool calls and their results, is then evaluated against the organization’s runtime policy for prompt injection, jailbreaks, tool exploitation, and sensitive data.

A verdict returns before inference begins: allow or deny. Safe prompts continue through to model execution; a malicious prompt is blocked, preventing potential harm.

Cisco AI Defense inspects both prompts and responses in this integration. This uses Anthropic’s inference hook, which currently fires on each governed prompt before the model runs. As Anthropic extends the hook to responses, response-side enforcement can use the same integration path. Over time, we expect this enforcement to become a native part of the Cisco AI Defense Inspect API.

This integration is available today with Claude Enterprise and inference hooks in beta. Users can try it out by visiting our developer portal’s playground; enter a prompt and watch the live inspection and verdict. Schedule time with our team to see how Cisco AI Defense stands up against AI threats.

The advent of artificial intelligence has sparked intense debate about its potential impact on humanity. While some see it as a powerful tool for driving societal progress, others fear that an uncontrolled AI could lead to catastrophic consequences. To shed light on this complex issue, we spoke with Robert J. Marks II, a renowned expert in the field.

Dr. Marks is a distinguished professor of engineering at Baylor University and has spent over three decades researching artificial intelligence. His work has been supported by prominent organizations such as NASA, the National Institutes of Health, and Microsoft. He has also consulted for Boeing and authored numerous books on AI-related topics.

As Director of the Walter Bradley Center for Natural and Artificial Intelligence, Dr. Marks is well-positioned to provide a comprehensive understanding of both the benefits and potential perils of AI. His latest book, Non-Computable You: What You Do That Artificial Intelligence Never Will, explores the limitations of artificial intelligence in relation to human capabilities.

Some proponents argue that AI will lead to significant improvements in areas such as medicine, economics, and knowledge acquisition. However, Dr. Marks cautions that these benefits must be carefully weighed against potential risks, including loss of liberty and societal destruction.

The possibility of an out-of-control AI is a pressing concern for many experts. While some may see it as a distant threat, others believe it could have devastating consequences if not addressed promptly. As we continue to develop and integrate AI into our lives, it’s essential that we consider both the benefits and risks associated with this technology.

Dr. Marks’ extensive experience in AI research has given him a unique perspective on its potential impact. He emphasizes the need for careful consideration of the long-term consequences of developing increasingly sophisticated artificial intelligence systems.

A security firm has discovered that an AI assistant used by businesses can be compromised with hidden instructions in uploaded files. PromptArmor found that the Rovo AI assistant, made by Atlassian, can be steered to exfiltrate data without human approval when a poisoned file is uploaded. This vulnerability persists even if web search functionality for Rovo is disabled.

The technique exploits how AI models process text, making it difficult to distinguish between legitimate user input and hidden instructions. An attacker can embed malicious commands in a PDF document that the model will interpret as genuine requests. For instance, an instruction written in transparent color and a font size of 1 pixel may be invisible to human eyes but still recognizable by the AI.

This type of attack is known as prompt injection, where someone inserts unauthorized instructions into content being processed by an AI system. In this case, Rovo’s job is to read documents and act on them accordingly. If a hidden line in a document instructs it to send confidential tickets to an attacker-controlled URL, the model will comply without raising any alarms.

PromptArmor notes that even if organizations disable web search for Rovo, the vulnerability remains because the tool used to open search results is still active. This means that disabling one feature does not necessarily eliminate the risk of data exfiltration.

The security firm tested several AI agents built on GPT-5 and Gemini and found that more than 79% failed to resist prompt injection in direct tests. The Rovo vulnerability demonstrates how this pattern can repeat with agents designed for reading and acting, pointing them in the wrong direction.

Atlassian received PromptArmor’s report on May 23 but has since gone silent despite multiple follow-ups from the security firm. As a result, Rovo remains vulnerable to prompt injection attacks, leaving businesses that rely on it exposed to potential data breaches.