Patreon
Home / Videos / 100% AI From a Text Prompt? All 37 Original OpenAI Sora Demo

100% AI From a Text Prompt? All 37 Original OpenAI Sora Demo Videos in 4K (February 2024)

Sora by OpenAI creates realistic and imaginative scenes from text; its 37 launch demo videos are merged into one 4K video, with every prompt listed.

Published Feb 16, 2024 · 6,630 views · Watch on YouTube

Get the app and files

Sora by OpenAI: New Text-to-Video AI Model, 37 Demo Videos & Prompts

Sora by OpenAI: New Text-to-Video AI Model, 37 Demo Videos & Prompts

Sora by OpenAI creates realistic and imaginative scenes from text; its 37 launch demo videos are merged into one 4K video, with every prompt listed.

About this video

Sora is an AI model that can create realistic and imaginative scenes from text instructions. It is just published by OpenAI today. The quality level is so good that I am super skeptical how can this be real. The quality, coherency, details, everything is just mind blowing. I have merged 37 demo videos they published in 4K quality.

Our Discord Channel ⤵️

https://discord.com/servers/software-engineering-courses-secourses-772774097734074388

Our Patreon With Amazing AI Scripts & Tutorials ⤵️

https://www.patreon.com/SECourses

Prompts Of Each Demo Video (Public Post) ⤵️

https://www.patreon.com/posts/98564118

Official Site ⤵️

https://openai.com/sora

Technical Report:

Video generation models as world simulators

We explore large-scale training of generative models on video data. Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios. We leverage a transformer architecture that operates on spacetime patches of video and image latent codes. Our largest model, Sora, is capable of generating a minute of high fidelity video. Our results suggest that scaling video generation models is a promising path towards building general purpose simulators of the physical world.

This technical report focuses on (1) our method for turning visual data of all types into a unified representation that enables large-scale training of generative models, and (2) qualitative evaluation of Sora’s capabilities and limitations. Model and implementation details are not included in this report.

Much prior work has studied generative modeling of video data using a variety of methods, including recurrent networks,1,2,3 generative adversarial networks,4,5,6,7 autoregressive transformers,8,9 and diffusion models.10,11,12 These works often focus on a narrow category of visual data, on shorter videos, or on videos of a fixed size. Sora is a generalist model of visual data—it can generate videos and images spanning diverse durations, aspect ratios and resolutions, up to a full minute of high definition video.

Turning visual data into patches

We take inspiration from large language models which acquire generalist capabilities by training on internet-scale data.13,14 The success of the LLM paradigm is enabled in part by the use of tokens that elegantly unify diverse modalities of text—code, math and various natural languages. In this work, we consider how generative models of visual data can inherit such benefits. Whereas LLMs have text tokens, Sora has visual patches. Patches have previously been shown to be an effective representation for models of visual data.15,16,17,18 We find that patches are a highly-scalable and effective representation for training generative models on diverse types of videos and images.

At a high level, we turn videos into patches by first compressing videos into a lower-dimensional latent space,19 and subsequently decomposing the representation into spacetime patches.

Video compression network

We train a network that reduces the dimensionality of visual data.20 This network takes raw video as input and outputs a latent representation that is compressed both temporally and spatially. Sora is trained on and subsequently generates videos within this compressed latent space. We also train a corresponding decoder model that maps generated latents back to pixel space.

Spacetime Latent Patches

Given a compressed input video, we extract a sequence of spacetime patches which act as transformer tokens. This scheme works for images too since images are just videos with a single frame. Our patch-based representation enables Sora to train on videos and images of variable resolutions, durations and aspect ratios. At inference time, we can control the size of generated videos by arranging randomly-initialized patches in an appropriately-sized grid.

In this work, we find that diffusion transformers scale effectively as video models as well. Below, we show a comparison of video samples with fixed seeds and inputs as training progresses. Sample quality improves markedly as training compute increases.

Base compute

Variable durations, resolutions, aspect ratios

Past approaches to image and video generation typically resize, crop or trim videos to a standard size – e.g., 4 second videos at 256x256 resolution. We find that instead training on data at its native size provides several benefits.

Music:

Song: Jim Yosef - Firefly [NCS Release]

Music provided by NoCopyrightSounds

Free Download/Stream: http://ncs.io/jyfirefly

Watch: http://youtu.be/x_OwcYTNbHs

Song: Egzod, Maestro Chives, Neoni - Royalty [NCS Release]

Music provided by NoCopyrightSounds

Free Download/Stream: http://ncs.io/Royalty

Watch: http://youtu.be/C5fLxtJH2Qs

Song: Lost Sky x ANGELPLAYA x Chris Linton - Unbreakable [NCS Release]

Music provided by NoCopyrightSounds

Free Download/Stream: http://ncs.io/Unbreakable

Watch: http://ncs.lnk.to/UnbreakableAT/youtube