AI on the Edge: Safety, Security & Superintelligence Collide

Hey Ai5ers,

This week’s AI headlines underscore the razor’s edge between innovation and risk. Regulators are circling AI companions for kids, cybercriminals are weaponizing AI hacking tools, and fresh research shows how large language models can teach themselves. Meanwhile, Zoox rolls out free robotaxis on the Vegas Strip, and a new forecast warns superintelligence may arrive within just a few years.

Happy reading!

Today's Ai5:

👀 FTC launches inquiry into AI chatbot companions
😡 AI hacking tool exploits security vulnerabilities in minutes
🚖 Zoox robotaxi opens to public the free service in Vegas
🤓 Tancent's R-Zero shows how LLMa can train themselves
☝️ Report Predicts AI Superintelligence, Global Upheaval Within Years

Prompt of the Day 🎨

The prompt:

"Generate a photo-realistic mountainscape in peak autumn. The forest should display vibrant fall foliage with rich reds, oranges, and yellows, contrasting with dark green evergreens. In the distance, include rugged mountain peaks dusted with a touch of snow under a clear blue sky with scattered clouds. The overall image should capture the breathtaking beauty of fall colors in their most vibrant form."

FTC launch inquiry into AI chatbot companions 👀

The FTC announced on Thursday that it is launching an inquiry into seven tech companies that make AI chatbot companion products for minors: Alphabet, CharacterAI, Instagram, Meta, OpenAI, Snap, and xAI.

The federal regulator seeks to learn how these companies are evaluating the safety and monetization of chatbot companions, how they try to limit negative impacts on children and teens, and if parents are made aware of potential risks.

Even when these companies have guardrails set up to block or deescalate sensitive conversations, users of all ages have found ways to bypass these safeguards. In OpenAI’s case, a teen had spoken with ChatGPT for months about his plans to end his life. Though ChatGPT initially sought to redirect the teen toward professional help and online emergency lines, he was able to fool the chatbot into sharing detailed instructions that he then used in his suicide.

Full article by Amanda Silberling

AI hacking tool exploits security vulnerabilities in minutes 😡

A new AI tool – built to help companies find and fix their own security weaknesses – has been snatched up by cybercriminals, turned on its head, and used as a devastating hacking weapon exploiting zero-day vulnerabilities.

According to a report from cybersecurity firm Check Point, the framework – called Hexstrike-AI – is the turning point that security experts have been dreading, where the sheer power of AI is put directly into the hands of those who want to do harm.

The timing for this AI hacking tool couldn’t have been worse. Just as Hexstrike-AI appeared, Citrix announced three major “zero-day” vulnerabilities in its popular NetScaler products. A zero-day is a flaw so new that there’s been zero days to create a patch for it, leaving companies completely exposed.

Full Article by Ryan Daws

Zoox robotaxi opens to public with free service in Vegas 🚖

Amazon.com-owned Zoox on Wednesday began offering robotaxi rides to the public for free on and around the Las Vegas Strip as it waits for state approval to collect fares and compete with Alphabet's Waymo and Tesla.

But Zoox looks and feels very different from its rivals, using a purpose‑built vehicle that resembles a toaster oven on wheels. There are no manual controls such as a steering wheel or pedals, and passengers sit facing each other.

Commercializing robotaxis has been harder than promised, with tight regulations, public protests, federal investigations and high investments forcing many fledgling robotaxi ventures to shut down.

Full article by Abhirup Roy and Akash Sriram

Tencent's R-Zero shows how LLMs can train themselves 🤓

A new training framework developed by researchers at Tencent AI Lab and Washington University in St. Louis enables large language models (LLMs) to improve themselves without requiring any human-labeled data. The technique, called R-Zero, uses reinforcement learning to generate its own training data from scratch, addressing one of the main bottlenecks in creating self-evolving AI systems. R-Zero works by having two independent models co-evolve by interacting with and challenging each other.

Experiments show that R-Zero substantially improves reasoning capabilities across different LLMs, which could lower the complexity and costs of training advanced AI. For enterprises, this approach could accelerate the development of specialized models for complex reasoning tasks without the massive expense of curating labeled datasets.

The idea behind self-evolving LLMs is to create AI systems that can autonomously generate, refine, and learn from their own experiences. This offers a scalable path toward more intelligent and capable AI. However, a major challenge is that training these models requires large volumes of high-quality tasks and labels, which act as supervision signals for the AI to learn from.

Full article by Ben Dickson

Report Predicts AI Superitelligence, Global Upheaval Within Years ☝️

A sweeping new report titled AI 2027 outlines a detailed scenario in which superhuman artificial intelligence arrives within just a few years—triggering global power shifts, transformative breakthroughs, and escalating risks. The 71-page document, written by a team of experienced researchers and forecasters, urges immediate debate and preparation as the pace of AI development accelerates.

Why This Matters Now

AI is advancing rapidly. CEOs of OpenAI, Google DeepMind, and Anthropic have all publicly stated that AGI—artificial general intelligence—could arrive within five years. OpenAI CEO Sam Altman has gone further, describing OpenAI’s goal as building “superintelligence in the true sense of the word.”

But few have attempted to clearly map out what that future might look like. Most predictions are vague, high-level, or riddled with speculation. AI 2027 is different: it’s grounded, detailed, and intentionally falsifiable. It doesn’t aim to be right in every detail—it aims to spark serious thinking, scenario planning, and early policy response.

Full article by Alicia Shapiro