This video was made from a single AI-generated image and will work with any suitable photograph.
We introduce VASA, a framework for generating lifelike talking faces of virtual characters with appealing visual affective skills (VAS), given a single static image and a speech audio clip. Our premiere model, VASA-1, is capable of not only producing lip movements that are exquisitely synchronized with the audio, but also capturing a large spectrum of facial nuances and natural head motions that contribute to the perception of authenticity and liveliness. The core innovations include a holistic facial dynamics and head movement generation model that works in a face latent space, and the development of such an expressive and disentangled face latent space using videos. Through extensive experiments including evaluation on a set of new metrics, we show that our method significantly outperforms previous methods along various dimensions comprehensively. Our method not only delivers high video quality with realistic facial and head dynamics but also supports the online generation of 512×512 videos at up to 40 FPS with negligible starting latency. It paves the way for real-time engagements with lifelike avatars that emulate human conversational behaviors.
This technology will work well in psychological games that use the faces of relatives and friends as well as fictional ones. The day will come when AI-generated characters will be able to play simple roles on their own. These roles will become more complex and creative as the tech evolves. Psychological games, especially when the user has a lot of control, will be fun and provide significant insights into how our minds are actually working. Gnarly psychologies, like narcissism or BPD, will benefit greatly from working through a variety of plots and scenarios.
Beyond self-help, the possibilities for deep works of art seem boundless. Whole novels with many variable plot-turns will raise the art of fiction to new levels of immersion and aesthetic reward. Coupled with brain-wave reading technology, AI characters will be capable of reacting to out mental and emotional states in ways no one has ever experienced before. I love where this is going. Of course there will be porn but the deepest transformative fun will be the realm of art and self-knowledge. ABN
American Buddhist Net 