Former Snap AI chief launches Higgsfield to compete with OpenAI’s Sora video generator

OpenAI mesmerized the tech world just a few months in the past with a generative AI mannequin, Sora, which transforms visible particulars into native video – no cameras or movie crew required. But Sora has been intently monitored to this point, and it appears the corporate is concentrating on well-funded creatives like hollywood director – Not essentially hobbyists or small-time entrepreneurs.

Alex Mashrabov, the previous head of generative AI at Snap, sensed a possibility. so that they launched higgsfield aiAn AI-powered video creation and modifying platform designed for extra personalized, personalised purposes.

Powered by a customized text-to-video mannequin, Higgsfield’s first app, Diffuse, can generate video from scratch or take a selfie and generate a clip starring that particular person.

“Our target audience is all types of creators,” Masherbov instructed TechCrunch in an interview, “from regular users who want to create fun content with their friends, to social content creators who want to try out a new content format on social media.” Even to entrepreneurs who need their model to face out.”

Mashrabov got here by the way in which to snap AI Factory, his earlier startup, which Snap acquired in 2020 for $166 million. While at Snap, Masherbov helped create merchandise equivalent to AR results and filters for Snapchat CameoAlso Snapchat’s controversial MyAI cabot

Higgsfield — which Masherbov co-launched a number of months in the past with Yerzat Dulat, an AI researcher specializing in generative video — gives a curated set of pre-made clips, a device for importing reference media (i.e. pictures and movies), and a Provides fast editor. Lets customers describe the characters, actions, and scenes they need to painting. Using Diffuse, customers can insert themselves straight into an AI-generated scene, or have their digital likeness mimic issues – like dance strikes – captured in different movies.

Image Credit: higgsfield

“Our model supports highly realistic movements and expressions,” Masherbov stated. “We are pioneering a ‘world model’ for consumers that will allow us to create best-in-class video generation and editing with the greatest level of control.”

Higgsfield is not the one generic video startup to go head-to-head with OpenAI. Runway was one of many first on the scene, and its instruments are always bettering. There’s additionally Haiper, which is backed by two DeepMind alumni and over $13M in enterprise money.

Mashrabov argues that Diffuse will stand out due to its mobile-first, social-forward go-to-market technique.

“By prioritizing iOS and Android apps over desktop workflows, we enable creators to create engaging social media content anytime, anywhere,” Mashrabov stated. “In fact, by building on mobile, we are able to prioritize ease of use and consumer-friendly features from day one.”

Higgsfield can also be operating weak. Masherbov says the generic mannequin underpinning the platform was developed by a 16-person staff in lower than 9 months and skilled on a cluster of 32 GPUs. (32 GPUs could look like lots, however provided that OpenAI makes use of 1000’s of them, it is not) In reality.) and Higgsfield have raised simply $8 million to this point, most of which got here from a current seed funding tranche led by Menlo Ventures.

Image Credit: higgsfield

To keep one step forward of rivals, Higgsfield plans to speculate early funding into constructing an improved video editor that may let customers modify characters and objects in movies, and extra particularly for social media use circumstances. Powerful video technology will permit coaching the mannequin. In reality, Masherbov sees social media – and social media advertising – as a money-making space of ​​Higgsfield’s idea.

While Diffuse is at the moment free to make use of, Masherbov envisions a future the place entrepreneurs pay some form of charge or subscription for premium options, or for quantity or large-scale campaigns.

“We believe Higgsfield opens up incredible levels of realism and content production use cases for social media marketers,” he stated. “We always hear from CMOs and artistic administrators that they should optimize content material manufacturing budgets and compress timelines whereas delivering impactful content material. So we imagine video generative AI options might be a key answer in serving to them obtain this.

Of course, Higgsfield isn’t any exception to the broader challenges going through generic AI startups.

It is effectively established that generative AI fashions equivalent to Powering can “reinvent” diffuse coaching knowledge. Why is that problematic? Well, if fashions have been skilled on copyrighted materials with out permission or some form of licensing settlement, customers of these fashions might unknowingly produce copyright-infringing work – which might depart them going through lawsuits.

Image Credit: higgsfield

Mashrabov wouldn’t disclose the supply of Higgsfield’s coaching knowledge (aside from to say that it got here from “several publicly available” areas), and likewise wouldn’t say whether or not Higgsfield would use person knowledge to coach future fashions. Maintain, which can not sit effectively with some enterprise clients. He famous that Diffuse customers can request to have their knowledge eliminated at any time by way of the app.

Digital “cloning” platforms like Higgsfield’s are additionally ripe for abuse, because the proliferation of deepfakes on social media in current months has proven.

Similarly, Higgsfield might make it simpler to steal creators’ content material. For instance, one want solely add a video of somebody’s choreography to create a video of themselves performing the identical choreography.

I requested Masherbov what safeguards or protections Higgsfield could be utilizing to attempt to stop abuse, and — although he would not go into particular particulars — he claimed the platform employs a mixture of automated and handbook moderation.

“We decided to introduce the product gradually and test it in select markets first, so we can monitor where there is potential for abuse and develop the product as needed,” Masherbov stated.

We’ll have to attend and see how effectively it really works in follow.