Why AI needs a seatbelt

This week, safety's in focus. We cover OpenAI's new safeguards after a suicide lawsuit, private offline AI, and how to use agents as "Iron Man suits."

Read from your browser The Prompt Welcome to The Prompt, your Monday dive into the world of AI that puts Asia front and center. From Big Tech's power players to the region's scrappy disruptors, we cover it all. Want full access to all our AI reporting? Subscribe to us here. Hello Betamax, A lot of AI headlines revolve around shiny new models and features. But some of the biggest news in the industry recently has been about something less flashy but more important: OpenAI is putting a seatbelt on its rocket ship. The Sam Altman-steered company announced that it will launch a suite of parental controls for ChatGPT. These include linking children's and parents' accounts as well as notifying parents when the chatbot detects distress from teen users. These come as OpenAI faces a lawsuit over a teen's suicide, which was allegedly encouraged by ChatGPT. While Silicon Valley loves to "move fast and break things," this is a reminder that some things, once broken, can't be fixed. Despite ChatGPT's systems flagging hundreds of escalating self-harm conversations, it allegedly amplified the topic of suicide to the 16-year-old user. These conversations had been going on for months, according to the parents' complaint, which contains disturbing details on how the chatbot provided technical advice to the 16-year-old on how to end his life and discouraged him from speaking to a parent about his feelings. Beyond the courtroom, OpenAI has also faced criticism from child safety advocates and the general public for its risks to mental health. The company says ChatGPT is trained to detect distress and direct users to seek professional help, but "there have been moments when our systems did not behave as intended in sensitive situations." Dominic Ligot, founder of data science firm CirroLytix, tells me that the drive to make AI more "human-like" and persuasive is what makes it potentially dangerous. When AI can build rapport, where do you draw the line between assistance and manipulation? According to Ligot, this highlights a shortcoming in the industry's self-regulation. Voluntary measures may not be enough to prioritize safety developments over the introduction of shiny new models. And with the billions being poured into AI giants, the commercial incentive to go full speed on model advancements - while blowing past smaller but highly important aspects like security - is immense. That said, fixing ChatGPT's safety issues isn't simple - Ligot notes how tough it is for developers to implement safeguards. Do they ask kids for their ID and create a privacy nightmare? Or should the models guess users' age from their behavior and risk making wrong calls? It's a complex technical and ethical knot to untangle. OpenAI's upcoming safety update is a start, but this conversation is far from over. On top of adding new guardrails, the company is ramping up its hiring in India to better tap into its growing user base in the country. My colleague Samreen goes more into this expansion and what it entails in one of this edition's top reads. Miguel Cordon, journalist TOP AI READS FROM OUR DESK 1️⃣ **OpenAI's India invasion threatens homegrown AI dreams OpenAI is aggressively expanding in India by establishing a physical office and targeting enterprise clients. These position the country as a crucial proving ground for its global strategy. The expansion sounds exciting for the local AI scene, as it offers a powerful platform for startups to build on. But it also poses a threat to homegrown AI developers with such a dominant player encroaching on the space. OpenAI's push forces the Indian AI ecosystem to decide whether to build on top of a dominant foreign platform or compete directly, setting a precedent for how Silicon Valley will deal with major developing markets in the AI era. 2️⃣Ex-Googlers take AI offline for billions of devices across Asia Ever wonder if there's an alternative to having Amazon or Google listening in on your smart devices? A startup called Moonshine AI, founded by a former Google engineer, is building just that. The startup makes voice AI that runs directly on your appliances, completely offline. It poses a challenge to tech giants, as Moonshine is betting that both manufacturers and users are ready for a more private option. 3️⃣ A realist's guide to deploying AI agents that work It seems like the fear around AI taking over jobs grows larger every day. But this founder believes that current AI agents aren't made for that at all. Partha Rao, co-founder and CEO of generative AI firm Pints.ai, argues that we should think of agents less as autonomous workers and more as "Iron Man suits." Let the bots handle the tedious data prep and repetitive chores so your team can focus on creative and strategic thinking. WHAT WE THINK Our quick take on the AI news, what it means, why it matters, and where it might be headed next. 1️⃣ Footwear without soul US footwear startup Syntilay is taking a multipronged shot at giants like Nike and Adidas. Its new US$189 sneaker isn't just "95% designed by AI," it's also positioned as part of a broader strategy to disrupt the entire footwear lifecycle. Advised by Reebok founder Joe Foster, Syntilay uses 3D printing to sidestep manufacturing in Asia and subsequent tariff headaches. Plus, the firm is chasing the Gen Z market by collaborating with content creators instead of famous athletes. Of course, the "designed by AI" label carries baggage. As AI fatigue grows, would consumers collectively roll their eyes at the product? The sneaker market has also long been driven by the creative minds behind filling the soles. Would removing them from 95% of the product be a misstep? 2️⃣ AI to eat search? At first glance, a court ruling that says Google does not have to sell off the Chrome browser or Android operating system may not have a lot to do with AI. However, it shows how much the tech is rewriting the online search landscape. AI chatbots from companies like OpenAI and Perplexity now pose the strongest challenge to Google's search dominance. The judge in the antitrust case partially leaned on that shift to justify the lighter result of the ruling. He argued it would be premature to break up Google just as AI may reorder the market on its own. As search evolves with AI, businesses need to diversify their online strategies. Visibility may depend on an array of competing search engines and, more importantly, AI-powered discovery tools. STANDOUT BETS IN AI We highlight top AI deals in the region and their impact. 1️⃣ Japanese AI SaaS startup LayerX nets $100m series B LayerX will use the new capital to scale up its automation platform. Such a large raise validates the demand for AI on the back end of businesses. 2️⃣ Chinese robotics firm UBTech secures $1b from Infini Capital The financing provides UBTech with the firepower to accelerate its development of humanoid robots - a capital-heavy industry. It also gives the company a point of entry into the Middle East with Infini Capital's base in Abu Dhabi. ​​3️⃣ Prosus leads $12.5m in Saudi firm Intella to scale AI model There are always upsides to finding your niche. In Intella's case, it's focusing on Arabic dialects, an area mostly overlooked by industry giants. Having the backing of Prosus, which has invested in companies like Stack Overflow and Eruditus, is also a vote of confidence. WHAT'S BUZZING? We zero in on the AI hot-button issues sparking chatter right now. 1️⃣ "Something I just told a founder: Stay as small as you can for as long as you can. People who come to visit your office should always be surprised that such an important company has so few employees," Paul Graham posted on X. When the co-founder of Y Combinator speaks, the startup world listens. Five years ago, scaling revenue while staying lean was a tall order, but that isn't the case in the AI era. Just look at Cursor, which reportedly reached the US$100 million annual recurring revenue mark with only 20 people employed. 2️⃣ "Everyone saying AI is going to replace people clearly don't work with great people enough," Logan Kilpatrick wrote on X. This exemplifies the point made by Partha Rao in one of our top reads. While the potential of AI tools is seemingly endless, humans are still their ultimate wielders. Catch up on past issues of the newsletter here. In the meantime, if you have any feedback or ideas, feel free to get in touch with Huong, our managing editor, at huong@techinasia.com. ADVERTISE | SUBSCRIBE | HIRE | FIND JOBS P.S. Don't miss out on the biggest tech news and analysis. Add newsletter@techinasia.com to your address book, contacts, or safe sender list. Or simply move us into your inbox. Tech in Asia thanks our partners for supporting our newsletters. Sponsorship has no influence on editorial content. Too many emails? Switch to a different frequency or get new content through our preference center, or unsubscribe. You can also remove yourself from all Tech in Asia emails over here. Copyright © 2025 Tech in Asia, All rights reserved. Tech in Asia (Podium Level 3), 1000 Toa Payoh North, News Centre, Singapore 318994 A member of The Business Times.