Connect with us

Tech

Microsoft Copilot can now read your screen, think deeply, and speak aloud to you

A week after announcing a wave of updates for its enterprise suite of Copilot AI-powered products, Microsoft is launching new Copilot capabilities on Windows for all users, including a tool that can understand and respond to questions about what’s on your screen.

Refreshed Copilot apps for iOS, Android, Windows and the web are rolling out today, and all feature a Copilot with a more “warm” and “distinct” style, as Microsoft describes it. Microsoft is also bringing the chatbot to WhatsApp, letting users chat with Copilot via DM, similar to the experience you get with other bots on Meta’s messaging platform.

Copilot Vision

Copilot Vision has a view of what you’re viewing on your PC — more specifically, a lens into the sites you’re visiting with Microsoft Edge. Gated behind Copilot Labs, a new, Copilot Pro-exclusive opt-in program for experimental Copilot capabilities, Copilot Vision can analyze text and images on webpages and answer queries (e.g., “What’s the recipe for the food in this picture?”) about them.

Vision, which can be pulled up by typing “@copilot” in Edge’s address bar, isn’t exactly a technical marvel. Google offers similar search technology on Android, and recently brought bits and pieces of that tech to Chrome as well.

But Microsoft suggests that Copilot Vision is more powerful and conscious of privacy than previous screen-analyzing features.

“Copilot Vision can … suggest next steps, answer questions, help navigate whatever it is you want to do, and assist with tasks, all while you simply speak to it in natural language,” Microsoft wrote in a blog post shared with TechCrunch. “Imagine you’re trying to furnish a new apartment. Copilot Vision can help you search for furniture, find the right color palette, think through your options on everything from rugs to throws, and even suggest ways of arranging what you’re looking at.”

Copilot Vision
Using Copilot Vision to ask questions about a photo on the web.
Image Credits: Microsoft

No doubt eager to avoid another round of bad press from AI privacy fumbles, Microsoft is stressing that Copilot Vision was designed to delete data immediately following conversations. Processed audio, images or text aren’t stored or used to train models, the company claims — at least not in this preview version.

Copilot Vision is also limited in the types of websites that it can interpret. For the time being, Microsoft’s blocking the feature from working on paywalled and “sensitive” content, limiting Vision to a pre-approved list of “popular” web properties.

What does “sensitive” content entail, exactly? Porn? Violence? At this juncture, Microsoft wouldn’t say.

Accusations of circumventing paywalls with AI tools have landed Microsoft in legal hot water in the recent past. In an ongoing lawsuit, The New York Times alleged that Microsoft allowed users to get around its paywall by serving NY Times articles through the Copilot chatbot on Bing. When prompted in a certain way, Copilot — which is powered by close Microsoft collaborator OpenAI’s models — would give verbatim (or close-to-verbatim) snippets of paid stories, according to The Times.

Microsoft said that Copilot Vision, which is U.S.-only at the moment, will respect sites’ “machine-readable controls on AI” — like rules that disallow bots from scraping data for AI training. But the company hasn’t said precisely which controls Vision will respect; there are several in use. We’ve asked Microsoft for clarification.

Many major publishers have opted to block AI tools from trawling their websites not only out of fear their data will be used without permission, but also to prevent these tools from sending their server costs soaring. If the current trend holds, Copilot Vision may not work on some of the web’s top news sites.

Microsoft said it’s committed to “taking feedback” to allay concerns.

“Before we launch broadly, we’ll continue to … refine our safety measures and keep privacy and responsibility at the center of everything we do,” Microsoft said in the blog post. “There is no specific processing of the content of a website you are browsing [with Copilot], nor any AI training — Copilot Vision simply reads and interprets the images and text it sees on the page for the first time along with you.”

Think Deeper

As with Vision, Copilot’s new Think Deeper feature is an attempt to make Microsoft’s assistant more versatile.

Think Deeper gives Copilot the ability to reason through more complex problems, Microsoft said, thanks to “reasoning models” that take more time before responding with step-by-step answers.

Which reasoning models? Microsoft was a bit cagey when I asked, saying only that Think Deeper uses “the latest models from OpenAI, fine-tuned by Microsoft.” Reading between the lines, it’s a safe bet that they’re a customized version of OpenAI’s o1 model.

“We’ve designed Think Deeper to be helpful for all kinds of practical, everyday challenges, like comparing two complex options side by side,” Microsoft wrote in a blog post. “Think Deeper can help with anything from solving tough math problems to weighing up the costs of managing home projects.”

Microsoft talked up Think Deeper’s potential quite a bit in its press materials. But assuming the model underneath is o1, it will most certainly fall short in some areas. We’re curious to see what sort of enhancements Microsoft made to the base model, and how forthcoming Think Deeper is about its limitations.

Think Deeper will be available from today to a limited number of Copilot Labs users in Australia, Canada, New Zealand, the U.S. and the U.K.

Copilot Voice

A new Copilot feature generally available today is Copilot Voice (not to be confused with GitHub’s Copilot Voice). Launching in English in New Zealand, Canada, Australia, the U.K. and the U.S. to start, Voice adds four synthetic voices, letting you talk to Copilot and have its responses be spoken aloud.

Copilot Voice
Image Credits: Microsoft

Like OpenAI’s Advanced Voice Mode for ChatGPT, Copilot Voice can pick up on your tone during conversations and respond accordingly, and you can interject at any point while Copilot Voice is answering. A Microsoft spokesperson told me that the mode uses “the latest voice technology with new models that have been fine-tuned for the Copilot app.” What tech? Which models? On the specifics, mum’s the word.

One thing to be aware of: Copilot Voice has a time-based usage limit. Copilot Pro subscribers get more minutes but the number is “variable,” Microsoft told me, depending on demand.

Personalization

Copilot will soon become more tailored to your likes and preferences, Microsoft said, thanks to a new personalization setting.

When the setting is enabled, Copilot will draw on your past interactions and history, as well as your interactions with other Microsoft apps and services (Microsoft won’t say which) to recommend ways to use Copilot.

“This helps you get going,” Microsoft wrote in a blog post, “offering both a handy guide to Copilot’s useful features and conversation starters.”

Personalization in Copilot, which can be switched off in the Copilot settings menu on Windows, isn’t slated for the U.K. or EU anytime soon. But users elsewhere should begin to see the setting this afternoon.

Microsoft and the EU have had a testy relationship where it concerns the company’s AI product rollouts. In May, the EU warned Microsoft that it could be fined up to 1% of its global annual turnover under the bloc’s online governance regime, the Digital Services Act, after the company failed to respond to a request for information that focused on its generative AI tools.

A number of tech giants beyond Microsoft, including Apple and Meta, have taken a cautious approach to launching AI tools in the EU, wary of running afoul of the bloc’s laws governing data privacy and model deployment.

“For users in the European Economic Area (EEA) and a limited number of other countries, we are evaluating options before offering this level of Copilot personalization for those users,” a Microsoft spokesperson told TechCrunch. “Some features will not be available in the EEA until a later date.”

source

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech

Founder of Shark Tank-backed startup Scholly sues his acquirer Sallie Mae

When Chris Gray sold his Shark Tank-backed scholarship search startup Scholly to Sallie Mae in 2023, he thought he had it all. Now he’s suing the student loan giant for wrongful termination and alleging that it’s selling the data his app collected, which includes personal info on minors, without properly informing users. 

Gray co-founded the company a decade prior with the hope of helping students more easily find college scholarships that were going untapped. Within two years, he nabbed sharks Daymond John and Lori Greiner as investors after an appearance on the show

With the acquisition, Gray became one of the few Black venture-backed fintech founders to exit their company, despite receiving some blowback that he was “selling out.” “I think being one of the first Black tech companies to get acquired by a bank, that’s really a big achievement,” he said at the time. 

He took a vice president role at Sallie Mae and expected to settle in nicely at his new gig, while helping scale Scholly and making it free to use, he said in an exclusive interview with TechCrunch.

What happened next is detailed in Gray’s lawsuit against Sallie Mae in Delaware Superior Court, and in a whistleblower complaint he submitted to the Securities and Exchange Commission, both of which he filed earlier this month. 

He alleges Sallie Mae laid off his employees, including his co-founders, and then went back on promises that it wouldn’t sell the users’ data, according to a TechCrunch review of both filings. He claims the company fired him a year after the acquisition when he tried to raise concerns about data privacy issues. Gray is seeking backpay and punitive damages in the suit, plus legal costs. 

Gray told TechCrunch that before he agreed to the sale, he believed Sallie Mae would be prohibited from disclosing or selling non-public personal information about Scholly customers to third parties because it was a federally regulated financial institution.  

Techcrunch event

San Francisco, CA
|
October 13-15, 2026

Now he alleges that his acquirer got around any such regulations by putting Scholly into a subsidiary that is selling the data — including age, gender, race, and other indicators of an individual’s financial need — to third parties like universities and advertisers, possibly without students’ full awareness.  

“I sold Scholly to a regulated bank because I believed it would protect the students who trusted us,” Gray told TechCrunch. “Instead, I watched the company build a non-bank subsidiary to do things the bank itself can’t legally do: sell student data. That’s not the company I thought I was joining.”  

Sallie Mae denied Gray’s allegations, calling them “without merit” and declined to answer TechCrunch’s questions about its data privacy practices. 

“While we don’t comment on pending litigation, it’s unfortunate a former employee is making false accusations about our company following his departure nearly two years ago. We plan to vigorously defend ourselves against these claims which are without merit or substance,” Rick Castellano, the company’s vice president of corporate communications, said in an email.  

Asked which specific accusations were “false,” Castellano declined to comment. 

From Alabama to Shark Tank

Gray grew up low-income in Birmingham, Alabama, with a single mother and two siblings. He felt the barriers to higher education were “real and immediate” for someone like him.  

Aside from being expensive, he felt he lacked access to information to help him make proper decisions about where to go and how to afford it, a pressure that only compounded after his mother lost her job in the 2008 recession.  

“That experience shaped how I thought about the scholarship system later,” he recalled, saying he began to view education and scholarship as “a problem of access rather than a problem of merit.”  

As a teenager, when the time came for him to apply for scholarships, he found the process fragmented and inefficient, he said. There was no centralized search for him to find opportunities, and when he did find a website with scholarship options, there were thousands of listings, but no reliable way to filter to see what he was actually eligible for. Not to mention the scams and outdated listings that persisted on some sites.  

Still, he applied to about 75 scholarships over the course of seven months using public computers and the internet at the library, and won around $1.3 million in scholarship funding, including from the Bill and Melinda Gates Foundation and the Coca-Cola Scholars Foundation.  

He studied economics and entrepreneurship at Drexel University and met students facing a familiar roadblock. “Students kept asking for help finding scholarships,” he told TechCrunch. “The funding existed with hundreds of millions of dollars unclaimed each year, but the search process was broken.”  

He started mapping out the eight core criteria that determined scholarship eligibility — age, location, major, GPA, race, gender, field of study, and financial need. 

“That became the foundation of Scholly’s matching algorithm,” he said.  

During his senior year, Gray, alongside Nick Pirollo and Bryson Alef, whom he met as Coca-Cola Scholars, officially launched Scholly in 2013. For just $0.99 a month, students could use the platform and filter by eligibility criteria. “That price kept the business sustainable without having to sell data or run ads,” he said.  

Scholly switched to a freemium model after Gray pitched the idea on Shark Tank. The sharks clamored over his idea in what became the “worst fight in Shark Tank history,” according to one of the hosts who invested. Scholly grew to 5 million users and made more than $30 million in cumulative revenue, Gray said. 

In March of 2023, Sallie Mae’s corporate development team reached out to Scholly. The bank had just bought the scholarship organization Nitro College a year prior and was trying to move more into the scholarship and college-planning space. “It was a natural fit,” Gray said, of why the student loan institution wanted Scholly.  

Sallie Mae bought Scholly in July 2023, brought Gray and his co-founders on board as employees, and made Gray a vice president of product management. 

In addition to promising that it would “make Scholly free for all students, families, and other users,” Sallie Mae CEO Jon Witter said in 2023 that the acquisition “allows us to harness and build on Scholly’s innovative technology to unlock future strategic growth opportunities.” 

Sallie Mae vs. “Sallie” 

For Gray, the canary in the coal mine came one year after Scholly’s acquisition.  

He alleges in the suit that Sallie Mae laid off the Scholly founding team, including his co-founders, in July 2024. Around this same time, Gray claims he heard Sallie Mae executives discuss plans for selling Scholly user data in meetings.  

Gray alleges executives told him his position was safe, and that the company was just restructuring. But when he went on to raise further concerns about the possible selling of Scholly data, he claims in his suit he was fired before a scheduled meeting with Witter, the CEO, where he planned to discuss those issues.  

After his departure, around December 2024, Sallie Mae launched “Sallie.com.” This website describes itself as an “education solutions company,” and became home to the Scholly platform. It is separate from the website for Sallie Mae, which is home to the bank that makes student loans. 

The Sallie.com website says it’s owned by an entity called SLM Education Services, LLC. Gray contends in his lawsuit and whistleblower complaint that Sallie Mae is using SLM Education Services in order to sell the personal data collected by Scholly, since it is not a closely regulated financial services company like the Sallie Mae banking arm. 

Sallie.com discloses that it sells the following customer data in its privacy policy to third parties: name, phone number, email addresses, age, race, gender, education records, and geolocation data. The third parties it sells this information to, it says, include ad networks, educational institutions, brands, and companies dedicated to reselling consumer data.  

Sallie Mae also pays Sallie “for the referrral of student loan customers,” according to the Sallie.com “About” page. 

Gray argues in his complaints that the Sallie.com website may be easily confused with the official Sallie Mae website because of similar layouts and “sallie” logos, increasing the risk that students may hand over personal data to what they believe to be a bank.  

Gray’s suit goes on to allege that Sallie Mae used Scholly user data to create something called Backpack Media in March, which it bills as a “first-to-market education media network” that “offers brands efficient, scalable access to highly desirable, hard to reach audiences – Gen Z, Gen Alpha, and those involved in their purchasing decisions,” according to a Sallie press release.  

Castellano declined to comment on Backpack Media’s sources for data.

This would not be the first time a Salle Mae-affiliated company has been accused of deceptive or misleading behavior.  

A company called Navient, which split from Sallie Mae in 2014, has faced restitution orders from the Federal Deposit Insurance Corporation, Department of Justice, and the Department of Education for overcharges. It was sued by the Consumer Financial Protection Bureau and reached a $1.85 billion settlement with 39 attorneys general for over what the attorneys general described as predatory student loans.  

Gray said he knew of these past legal issues, but that he doesn’t regret the sale of Scholly as it helped make the platform free for every student. In fact, he said if he could, he would make the same decision to sell all over again. 

“But I’d also raise the same concerns again,” he said. “Because I believe we should live in a system where an executive can speak up and change the course of a company in line with the law and fair business practices.”

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading

Tech

Lovable launches its vibe-coding app on iOS and Android

Apple’s recent crackdown on vibe-coding apps hasn’t held up Lovable’s launch of its no-code AI app builder, which is now available as a mobile app on Apple’s and Google’s app stores.

The vibe-coding startup’s new mobile app is being pitched to would-be app builders as a way to code on the go via voice or text AI prompts that let you capture your ideas as they pop into your head. That means you can kick off Lovable to work on your random app idea from anywhere, letting its agent run autonomously after receiving your input.

The new app will also allow you to switch back and forth between your computer and phone to pick up where you left off on a given project and receive notifications when a build is ready for review.

The app’s arrival comes shortly after Apple addressed what vibe-coding apps can and can’t do on its App Store. The tech giant recently blocked updates to popular vibe-coding tools, including Replit and Vibecode, for violations of its developer guidelines.

Simply put, Apple wasn’t banning vibe-coding apps themselves, but it won’t allow apps that download new code or change their functionality, as that presents a security risk to end users. (It also means that Apple’s App Review team can’t properly vet the app during the approval process.)

Apple also temporarily removed the vibe-coding app Anything from the App Store for similar reasons, but the app returned after making changes earlier this month.  

To comply with Apple’s rules, the vibe-coding apps are no longer able to run their generated apps inside the host app. Instead, those app previews were moved to web browsers.

Lovable has also seemingly complied with these rules as its new app touts the ability to turn ideas into “working websites or web apps.”

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading

Tech

Australia forces Big Tech firms to pay for news or face a 2.25% tax

Australia is getting serious about making Big Tech pay for news. The country’s government unveiled draft legislation on Tuesday that would require companies like Meta, Google, and TikTok to pay for the journalism they aggregate or reshare, or face a levy on their local revenues.

Communications minister Anika Wells said at a press conference today: “People are increasingly getting their news directly from Facebook, from TikTok, and from Google.”

The proposed law, called the News Bargaining Incentive (NBI), would impose a 2.25% levy on the Australian revenues of the three platforms unless they strike commercial deals with local news publishers. Plus, the more deals they make with media outlets, the less they pay. If enough agreements go through, that effective rate drops to 1.5%, which could generate between A$200 million and A$250 million back into Australian journalism.

“Journalists are the lifeblood of Australia’s media sector, playing a vital role in keeping communities informed about the news that matters to them,” Prime Minister Anthony Albanese said in a statement.

It is the country’s second attempt to force Big Tech to fund journalism. The Australian government introduced the News Media Bargaining Code, which officially came into effect in 2021, requiring platforms like Google and Meta to pay news publishers. But the original version had a flaw that Big Tech companies could simply remove news from their platforms to avoid paying. Meta did that in 2024, and that move, reportedly, triggered widespread job cuts across Australian newsrooms.

Meta’s decision to pull news content in 2024 left a pretty obvious gap in Australia’s media rules. The NBI is the government’s attempt to fix it, and this time, there’s no workaround. Platforms get taxed whether they carry news or not. The Albanese government first announced the NBI in December 2024 as a replacement for the existing 2021 Code, and the draft legislation finally landed today.

TikTok’s inclusion marks a notable expansion from the Code. And the draft legislation explicitly excludes AI services. Assistant treasurer Daniel Mulino said at today’s press conference that AI “is not included in the scope of this measure” because “AI is currently being examined through a range of other policy forums, including, for example, the work on copyright being led by the Attorney-General.”

Techcrunch event

San Francisco, CA
|
October 13-15, 2026

The Trump administration has consistently opposed digital services taxes on U.S. tech companies, repeatedly threatening tariffs against countries that push ahead with them. Most recently, Trump has warned the U.K. that it could face steep tariffs unless London drops its digital services tax on U.S. tech giants that derive value from British users, including Google, Meta, and Apple.

When a journalist asked about the pushback from the White House, Albanese said at the press conference, “We’re a sovereign nation, and my Government will make decisions based upon the Australian national interest. We do that right across the board.”

If passed in Australia, platforms have until July to comply, the same date the levy kicks in.

Australia isn’t alone in this fight. Canada, Brazil, and the EU have all taken on Big Tech over news, with mixed results. Canada’s 2023 law prompted Meta to pull news from its platform entirely. Brazil’s bill has been stuck in legislative limbo since 2019. The EU has rules on the books, but enforcement varies widely. South Africa may offer the clearest blueprint — regulators there brokered direct deals with Google, Meta, TikTok, and Microsoft, securing roughly $40 million for local news outlets over five years.

Meta, Google, and TikTok did not immediately respond to requests for comment.

When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.

source

Continue Reading