I dipped my toe in the water with AI last year and was pretty impressed with the image generation from text prompts. I wondered then how long it would be before generative AI video would come for my job. Since then I've seen what Sora can do and if it's as good as it looks (and it will only get better), I'm guessing it won't be long now before a lot of web based video content creation will go this way.
Sora isn't widely available for use yet but Runway Gen-2 is. So, I thought I'd take a look and see what I can get from the free account.
I used a lot of different prompts with varying degrees of success and made a little reel showing the results...
Obviously it's still got a long way to go, especially when you look closely at faces and the edges of frame but it shows incredible potential. I particularly liked the pirate ship and the way the water moves for the river kayaker. I also thought the hippies on a 1960s Carnaby street were pretty good. I used a variety of styles and prompts but mostly tried to include people.
You get 125 seconds of video generation with a Runway Gen-2 free account and each clip it generates is 4 seconds long based on a text prompt you enter. You can extend a clip to 16 seconds but I didn't bother with that. Another thing you notice right away is that the clips all appear to be slow motion (I'd say about 50-100 fps). This might be because it is easier for an AI to interpolate movement at this speed, as differences between frames are reduced. I don't know. You can also see that it struggles with distortion, particularly on hands and faces. I should also add that for all of my prompts I added camera movement, so that probably made it a little harder for the AI.
Overall though, you can see where this is heading already.
One final note, the music is also AI generated using Suno.
Great work, and I really like the AI soundtrack!