I just listened to this AI generated audiobook and if it didn’t say it was AI, I’d have thought it was human-made. It has different voices, dramatization, sound effects… The last I’d heard about this tech was a post saying Stephen Fry’s voice was stolen and replicated by AI. But since then, nothing, even though it’s clearly advanced incredibly fast. You’d expect more buzz for something that went from detectable as AI to indistinguishable from humans so quickly. How is it that no one is talking about AI generated audiobooks and their rapid improvement? This seems like a huge deal to me.
I’m only shocked that video isn’t better. Diffusion models work like denoising - so you’d figure all the wiggly nonsense between frames would be the first thing to filter out.
I expect the data size to be a problem. Stable diffusion defaults to 512x512px, because it simply requires a lot of resources to generate an image. Even more so to train one. Now do that times 30 to generate even one second of video. I think we need something that scales better.
I fully expect this to work decently in a few years though, no matter how hard the challenge is, ai is moving really fast.
I give it a year, maybe two, for a fully synthetic video that couldn’t not be easily distinguish from reality. There’s already some very good AI that complete or replace backgrounds on videos that work really good, and completely synthetic videos that looks like nightmares for now.
I expected it to be here six months ago, but its continued absence hasn’t changed my estimate from “any day now, and suddenly.” All of this is so weirdly democratized (and pornography-motivated) that we’re seeing the cool stuff before all the scary disinformation concerns.
And the underlying mechanisms are straight-up “the missile knows where it is, because it knows where it is not.” Stable Diffusion compares the noise estimate with and without a particular term, takes the difference, and then leaps outward along that vector.