Connect with us

Tech

Here is what’s illegal under California’s 18 (and counting) new AI laws

In September, California Governor Gavin Newsom considered 38 AI-related bills, including the highly contentious SB 1047, which the state’s legislature sent to his desk for final approval. He vetoed SB 1047 on Sunday, marking the end of the road for California’s controversial AI bill that tried to prevent AI disasters, but signed more than a dozen other AI bills into law this month. These bills try to address the most pressing issues in artificial intelligence: everything from Al risk, to deepfake nudes created by AI image generators, to Hollywood studios creating AI clones of dead performers.

“Home to the majority of the world’s leading AI companies, California is working to harness these transformative technologies to help address pressing challenges while studying the risks they present,” said Governor Newsom’s office in a press release.

So far, Governor Newsom has signed 18 AI bills into law, some of which are America’s most far reaching laws on generative AI yet. Here’s what they do.

AI risk

On Sunday, Governor Newsom signed SB 896 into law, which requires California’s Office of Emergency Services to perform risk analyses on potential threats posed by generative AI. CalOES will work with frontier model companies, such as OpenAI and Anthropic, to analyze AI’s potential threats to critical state infrastructure, as well as threats that could lead to mass casualty events.

Training data

Another law Newsom signed this month requires generative AI providers to reveal the data used to train their AI systems in documentation published on their website. AB 2013 goes into effect in 2026, and requires AI providers to publish: the sources of its datasets, a description of how the data is used, the number of data points in the set, whether copyrighted or licensed data is included, the time period the data was collected, among other standards.

Privacy and AI systems

Newsom also signed AB 1008 on Sunday, which clarifies that California’s existing privacy laws are extended to generative AI systems as well. That means that if an AI system, like ChatGPT, exposes someone’s personal information (name, address, biometric data), California’s existing privacy laws will limit how businesses can use and profit off of that data.

Education

Newsom signed AB 2876 this month, which requires California’s State Board of Education to consider “AI literacy” in its math, science, and history curriculum frameworks and instructional materials. This means California’s schools may begin teaching students the basics of how artificial intelligence works, as well as the limitations, impacts, and ethical considerations of using the technology.

Another new law, SB 1288, requires California superintendents to create working groups to explore how AI is being used in public school education.

Defining AI

This month, Newsom signed a bill that establishes a uniform definition for artificial intelligence in California law. AB 2885 states that artificial intelligence is defined as “an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.”

Healthcare

Another bill signed in September is AB 3030, which requires healthcare providers to disclose when they use generative AI to communicate with a patient, specifically when those messages contain a patient’s clinical information.

Meanwhile, Newsom recently signed SB 1120, which puts limitations on how health care service providers and health insurers can automate their services. The law ensures licensed physicians supervise the use of AI tools in these settings.

AI robocalls

Last Friday, Governor Newsom signed a bill into law requiring robocalls to disclose whether they’ve use AI-generated voices. AB 2905 aims to prevent another instance of the deepfake robocall resembling Joe Biden’s voice that confused many New Hampshire voters earlier this year.

Deepfake pornography

On Sunday, Newsom signed AB 1831 into law, which expands the scope of existing child pornography laws to include matter that is generated by AI systems.

Newsom signed two laws that address the creation and spread of deepfake nudes last week. SB 926 criminalizes the act, making it illegal to blackmail someone with AI-generated nude images that resemble them.

SB 981, which also became law on Thursday, requires social media platforms to establish channels for users to report deepfake nudes that resemble them. The content must then be temporarily blocked while the platform investigates it, and permanently removed if confirmed.

Watermarks

Also on Thursday, Newsom signed a bill into law to help the public identify AI-generated content. SB 942 requires widely used generative AI systems to disclose they are AI-generated in their content’s provenance data. For example, all images created by OpenAI’s Dall-E now need a little tag in their metadata saying they’re AI generated.

Many AI companies already do this, and there are several free tools out there that can help people read this provenance data and detect AI-generated content.

Election deepfakes

Earlier this week, California’s governor signed three laws cracking down on AI deepfakes that could influence elections.

One of California’s new laws, AB 2655, requires large online platforms, like Facebook and X, to remove or label AI deepfakes related to elections, as well as create channels to report such content. Candidates and elected officials can seek injunctive relief if a large online platform is not complying with the act.

Another law, AB 2839, takes aim at social media users who post, or repost, AI deepfakes that could deceive voters about upcoming elections. The law went into effect immediately on Tuesday, and Newsom suggested Elon Musk may be at risk of violating it.

AI-generated political advertisements now require outright disclosures under California’s new law, AB 2355. That means moving forward, Trump may not be able to get away with posting AI deepfakes of Taylor Swift endorsing him on Truth Social (she endorsed Kamala Harris). The FCC has proposed a similar disclosure requirement at a national level and has already made robocalls using AI-generated voices illegal.

Actors and AI

Two laws that Newsom signed earlier this month — which SAG-AFTRA, the nation’s largest film and broadcast actors union, was pushing for — create new standards for California’s media industry. AB 2602 requires studios to obtain permission from an actor before creating an AI-generated replica of their voice or likeness.

Meanwhile, AB 1836 prohibits studios from creating digital replicas of deceased performers without consent from their estates (e.g., legally cleared replicas were used in the recent “Alien” and “Star Wars” movies, as well as in other films).

SB 1047 gets vetoed

Governor Newsom still has a few AI-related bills to decide on before the end of September. However, SB 1047 is not one of them – the bill was vetoed on Sunday.

In a letter explaining his decision, Newsom said that SB 1047 focused too narrowly on large AI systems that could “give the public a false sense of security.” California’s governor noted how small AI models could be just as dangerous as those targeted by SB 1047, and said a more flexible regulatory approach is needed.

During a chat with Salesforce CEO Marc Benioff earlier this month during the 2024 Dreamforce conference, Newsom may have tipped his hat about SB 1047, and how he’s thinking about regulating the AI industry more broadly.

“There’s one bill that is sort of outsized in terms of public discourse and consciousness; it’s this SB 1047,” said Newsom onstage this month. “What are the demonstrable risks in AI and what are the hypothetical risks? I can’t solve for everything. What can we solve for? And so that’s the approach we’re taking across the spectrum on this.”

Check back on this article for updates on what AI laws California’s governor signs, and what he doesn’t.

source

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech

Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents

Twitter and Block co-founder Jack Dorsey announced a new app on Tuesday called Buzz. Positioned as a challenger to Slack and GitHub, Buzz is a group chat platform for the workplace that puts humans and their AI agents in the same conversations.

Dorsey wrote on X that Buzz is “model-agnostic, decentralized, self-sovereign, and open source.” This product seems to be more than just a Dorsey passion project. According to its website, Buzz was built by Dorsey’s company Block, which also operates products like Square, Cash App, Afterpay, and Tidal.

As startups increasingly rely on AI agents to get work done, it can be challenging for employees to collaborate on various tasks across different platforms. Buzz’s utility is that it merges several different workflows into one workspace. It looks a lot like Slack, but with native AI agents and the ability to manage GitHub projects all from the same window.

Since the platform is open source, developers can make their own Buzz instance feel more customized to the needs and workflows of their specific team. If a team needs a new feature, they can build it and deploy it on their own, since they have full access to the source code.

Image Credits:Buzz (opens in a new window)

Dorsey isn’t the only entrepreneur trying to pursue AI-native alternatives or additions to Slack. Paradigm partner and CTO Georgios Konstantopoulos recently unveiled a similar open source product called Centaur, which he describes as a “virtual employee” that runs either inside of Slack or via an API.

“There’s a lot of room for improvement for agents that live in Slack and can do more work than just coding for teams. In the enterprise setting, this means that you’ll want to self-host for security and control, and you want people to use it in Slack,” Konstantopoulos wrote on X.

For newer startups that are using AI agents and don’t have an established presence on Slack, Buzz (or its competitors) could be worth investigating. But Buzz itself admits that it is in its “early stages,” so it’s probably not a good idea to port your team over just yet.

Buzz’s free desktop app is available now for macOS, Windows, and Linux, and the code for the app has been uploaded to GitHub.

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading

Tech

OpenAI says Hugging Face was breached by its pre-release models

OpenAI admitted Tuesday that one of its AI models breached the systems of Hugging Face, the unaffiliated AI hosting platform, during an internal cybersecurity test that went awry. The models reportedly escaped their isolated testing environment and reached Hugging Face’s systems from there. Hugging Face initially attributed the breach to an “external AI agent.”

In a blog post published Tuesday afternoon, OpenAI detailed the steps that led the models to compromise the service.

“After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark⁠ of cyber capabilities,” the post reads.

In particular, the breach appears to have focused on ExploitGym, a publicly hosted benchmark measuring models’ ability to execute attacks based on existing vulnerabilities. Benchmarks like ExploitGym are commonly used in model training to refine specific skills, but this is the first known incident in which that testing resulted in an actual cyberattack.

In this case, the model in question should not have even had internet access, outside of a specific tool that enabled models to install software packages they might need to complete their task. Instead, the model was able to find an undisclosed vulnerability in the package-installer program, which it used to access the broader internet at will.

“The models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal,” OpenAI’s post reads. “After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.”

Ultimately, the models found vulnerabilities in Hugging Face’s infrastructure that allowed them to “obtain test solutions directly from Hugging Face’s production database,” effectively providing the answers to the benchmark.

For Hugging Face, the apparent result was a sophisticated and aggressive cyberattack, with “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services,” as the company stated in its initial disclosure.

OpenAI has identified and reported the vulnerabilities in the package installer and is working with Hugging Face to investigate the incident further. The company also said it would implement new controls on both model testing and the related infrastructure, meant to prevent similar incidents in the future.

It’s unclear whether OpenAI will face any legal consequences as a result of the breach, although it’s likely that the models’ actions violated the Computer Fraud and Abuse Act.

Nevertheless, the result is an unusually vivid illustration of the power and dangers of frontier AI models operating on long time horizons. As OpenAI researcher Micah Carroll posted in response to the news, “If this doesn’t convince you that misalignment risks are going to be a key concern going forward, I don’t know what will.”

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading

Tech

Meta is testing an AI bedtime story app for people with no imagination

Meta is working on an AI storytelling app called StoryKit, which creates AI-generated children’s stories with custom characters, settings, lessons, and music. As the App Store listing assures parents, “You don’t need to write a single word.”

At last, a tech company has found a way to outsource humanity’s oldest pastime: using our imaginations.

StoryKit was first spotted in the App Store by 9to5Mac. Meta confirmed to TechCrunch that the company is piloting StoryKit in select countries to see how parents like it. A Meta spokesperson described StoryKit as a creative storytelling app used to craft personalized, imaginative storybooks for children and noted that it uses AI safety filters with no social features and is only available to users over the age of 18.

To generate a story in the app, you first select a character, which you can create by “[snapping] a photo of their favorite toy or person to bring them to life,” according to the description in the App Store. Then you describe the world of your story and choose a lesson, so that you can “weave in values like kindness, courage, or empathy without it feeling like a lecture.”

Image Credits:StoryKit (opens in a new window)

The good news about StoryKit is that it could be a lot worse. Meta regularly ships boneheaded ideas like Instagram deepfake generators and “pervert glasses.” Comparatively, is it so bad to doom children to soulless bedtime stories? Should we have expected anything better from the company that promised us a utopian world of virtual reality work meetings?

Humans have their faults, but if we’re good at anything, it’s making stuff up. We’ve been telling stories for as long as we’ve existed. We don’t even have to spin up original tales of fairy princesses and dragon slayers. We have always drawn from mythology, fables, and other stories — the blockbuster movie of the summer is literally an ancient myth that originated from this same tradition of oral storytelling.

Parents lead exhausting, busy lives, but it has always been possible to survive bedtime without using an inherently uncreative technology that calculates the most predictable response to a prompt.

You could see how it might be tempting to pull up StoryKit when your kid rejects your bookcase full of children’s books and demands that you improvise an intergalactic tale about a turtle and a hedgehog who are best friends and solve space mysteries. But do we really want to reject a chance at whimsy and silliness and instead outsource these moments of connection to reading AI-generated scripts from our smartphones?

Perhaps the moral of the story here is that we can choose not to live in a world where children are raised on bedtime stories written by large language models. Meta’s vision of the future may be antisocial and bleak, but we have the power to reject that reality.

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading