The AI discussion thread

[Kaido]

Ahh man here we go...AI for animation! The labor reduction this technology is going to offer is going to open the doors for some REALLY incredible animated movies in the future!

https://twitter.com/x/status/1698520099739582902

I got heavily into filming & editing in grade school...back in the 90's, all of the processes were so expensive & complicated that a lot of techniques were simply out of reach. MiniDV (digital tape for home) came out in 1995 & it was SUCH a chore to do editing at home on a computer, but at least you could do it! Now you can get things like the free LiveLinkFace app to do real-time motion-capture animation in the Unreal game engine:

Which, when combined with AI video overlay generation, can take previous iterative jumps like Deepfakes to the next level. Remember this video by Jordan Peele a few years ago?

https://youtu.be/cQ54GDm1eL0

So now you've got crazy stuff like Gen-1 from Runway Labs:

Gen-1 by Runway

The Next Step Forward for Generative AI

research.runwayml.com

Scroll down the website & use the sliders to see examples of stylistic overlays on existing video footage:



This is where it starts getting scary...right now it's doing motion-video replacement using text prompts, but generative-fill & spot-editing are going to be the next tools released:



And you don't even have to do motion-capture anymore...you can use untextured renders as the source material to apply design styles to the finished images & videos. The power this has for cinematic storytelling & the doors it will open to end-users who don't have high-end budget or resources is going to be AMAZING!



Runway's Gen-2 takes that to the next level with video synthesis from text, images, and video clips (i.e. not just text prompts as the source anymore!). Scroll through the examples here:

Gen-2 by Runway

A multimodal AI system that can generate novel videos with text, images or video clips.

research.runwayml.com

Because computers excel at math, brush-based vector artwork (ex. digital art from say Illustrator & Procreate) can be animated with EXTREMELY good results:

September 4, 2023 - Runway Gen-2 Testing | Image and text description

Experiment No: 32Midjourney 5.2, Runway Gen-2

youtu.be

Plus with Gen-2, now you can do things like take an actual photo & not only animate it, but also stylize it & use generative fill to change the background, lighting, etc. (check out the animation of the video below on the link above)



This is all in the VERY early stages of development, so the quality is low, but soon, it will be impossible to tell what's real in videos anymore, which is REALLY scary! What gets even CRAZIER is thanks to the mobile supercomputers in our pocket, they now have a smartphone app for doing stylistic overlays in real-time! Imagine walking around with a VR headset outside & just living in a completely different world, Ready Player One-style!

Introducing: Runway for Mobile

The magic of Gen-1. Now on your phone. Download the new Runway iOS app today: https://apple.co/41zhB5BWant more helpful tutorials and video content? Follow u...

youtu.be

This is going to open the doors to a WORLD of amazing films & animated movies for people who don't have budgets in the millions or vast arrays of supercomputer rendering farms available!

 

[Kaido]

Pandora's box has now been unleashed...what have we done lol

https://imgur.com/a/t1y3uta

 

[Kaido]

Animation sample 100% from Generative A.I.:

Whaaaaaaaaaaat the heck man

https://twitter.com/x/status/1698706938513801343

 

[Kaido]

Using AI to create comic books:

https://twitter.com/x/status/1689264484878008320

More detailed information:

Consistency of Characters, Objects, and Styles with Generative AI

AI Tutorial

javilopen.substack.com

 

[Kaido]

The artistic stuff is really blowing up right now...you can now train AI to replicate specific art styles:

https://twitter.com/x/status/1697052500572913806

 

[Kaido]

Here's some next-level creepy stuff...training AI on poorly-lit vintage photos to generate new images in that style:

https://twitter.com/x/status/1696138560565641235

 

[JEDI]

Animation sample 100% from Generative A.I.:

Whaaaaaaaaaaat the heck man

https://twitter.com/x/status/1698706938513801343

holy christ i would have guessed a Ghibli film.
His animation films are usually completely hand drawn and takes thousands of man hrs but makes $Billions$ of Yen.

I guess since he's 85 and retired, he's probably not going to live long enough to see his beloved Studio crushed

 

[Kaido]

His animation films are usually completely hand drawn and takes thousands of man hrs but makes $Billions$ of Yen.

I think they'll figure out by:

1. Incorporating talented hand-animators
2. Who use (future) refined AI Assistive technologies

Hayao Miyazaki experimented with hybrid CGI in his short film "Boro The Caterpillar"

They still did stuff by hand:



But incorporated CGI into the final product: (unfortunately, the film is only available to view at their museum!)



Miyazaki’s son Goro went on to direct a 3D CGI movie called "Earwig and the Witch", which had a decent story but looked goofy, like those cheapo knockoff Youtube CGI videos for kids:

Hayao Miyazaki Stays Dedicated to Hand-Drawing: ‘I Believe the Tool of an Animator Is the Pencil’

Miyazaki is still in production on “How Do You Live?,” his first feature since announcing his retirement in 2013.

www.indiewire.com

Goro also did a cel shading project on Netflix called Ronja, which had a pretty fun story, but had that weird 2D-CGI look & stuttery framerate effect, which kept throwing me out of the story:

https://youtu.be/sVL6fhCA3_E

Spirited Away actually used some limited CGI:

How Spirited Away Changed Animation Forever

Spirited Away turns 20 this year. TIME looks at its path from Japanese blockbuster to Oscar winner to beloved classic

time.com

The problem is both manpower (time) & money:

In an interview with Entertainment Weekly last May, Studio Ghibli’s Suzuki talked about the hand-drawing process for Miyazaki’s upcoming film, How Do You Live? “We have 60 animators, but we are only able to come up with one minute of animation in a month,” he said. “That means 12 months a year, you get 12 minutes worth of movie.” It’s a painstaking process, but one that has undeniably shaped the singular animation aesthetic of Miyazaki films.

Especially because there are so many creative design decisions required for nuanced animation:

Behind The Scenes- Spirited Away

youtu.be

If you can use AI to come up with the design language for the AI model to generate & then use motion capture, prompt engineers, and a talented creative director who works with voice actors, design stylists for the set & costumes, and so on, I think they'd be able to create some REALLY brilliant animated films on a more feasible budget & timeline. On the manual side of things, they're already doing a really good job blending 2D animation with 3D CGI, such as on One Punch Man, Season 2: (which unfortunately had subpar animation to begin with, as compared to Season 1)

Reddit - Dive into anything

www.reddit.com

AI is inevitable. As an artist, it has me torn: it's scary that you can just push a button to generate art & that you can mimic anyone's art through cloning tools. What makes art & music fun is the unique human inspiration behind it. On the flip side, the ability to generate music, writing, soundtracks, singing, voice acting, illustration, art, "photography", video, and animation on your home computer (or even your smartphone) is not only remarkable but SUPER useful! Especially when applied to business needs in the real world, like how slow hand-animation is & how expensive it is & how much stress it puts on the artists to deliver on the gun of a tight deadline.

My minor in school was in filmmaking & I think AI is going to open up some really bonkers stuff for people to create really visually incredible stories. Virtual production using real-time game engines coupled with LED panels has already become the standard in Hollywood thanks to COVID:

The technology that’s replacing the green screen

The green screen is a Hollywood staple. Should it be?It’s easy to complain about overreliance on special effects, but for projects that require impossible-to...

youtu.be

The Virtual Production of The Mandalorian Season Two

A behind the scenes look at the groundbreaking virtual production technology used on The Mandalorian, season two.For the second season of Lucasfilm’s hit Dis...

youtu.be

Virtual Production Explained — Is This The End of the Green Screen?

What is Virtual Production — an introduction to the latest filmmaking technology, including how virtual production works, the pros and cons, and why it may b...

youtu.be

The Mandalorian looked really good for just being a TV show:

Why 'The Mandalorian' Uses Virtual Sets Over Green Screen | Movies Insider

For decades, film and TV productions have used green and blue screens to place actors into new environments. Now, LED walls are revolutionizing this process ...

youtu.be

So while AI is scary stuff, the cat is out of the bag & progress is in rapid development...people are afraid of it taking over, but really, it's just going to be another tool to help us get stuff done in a faster, better, and more efficient way!

 

[Kaido]

Researchers say it’s virtually impossible to make an A.I. model ‘forget’ the things it learns from private user data:

A.I.’s un-learning problem: Researchers say it’s virtually impossible to make an A.I. model ‘forget’ the things it learns from private user data

As it turns out, it’s impossible to remove a user’s data from a trained A.I. model. Deleting the model entirely is also difficult—and there’s little regulation to enforce either option.

finance.yahoo.com

 

[igor_kavinski]

Researchers say it’s virtually impossible to make an A.I. model ‘forget’ the things it learns from private user data

Rubbish! Someone will come up with memory wipe functionality sooner or later

Whenever someone says impossible, others take it on as a challenge.

 

[Kaido]

Rubbish! Someone will come up with memory wipe functionality sooner or later

Whenever someone says impossible, others take it on as a challenge.

I mean, just think about how many old files you have on your desktop, google drive, email, etc. Now imagine everyone who is feeding data into Chat-GPT's mega-vacuum on privately-held servers that they have no control over...there's just going to be loads & loads of stuff out there forever!

 

[Kaido]

RIP Smashmouth's lead singer. An AI tribute:

 

[Kaido]

AI portraits with analog film emulation:

https://twitter.com/x/status/1699075131778461762

Long exposure film emulation:

https://www.reddit.com/r/midjourney/comments/14pzi1l

 

[Kaido]

Here's an amazing application of AI:

1. You record a video for a few minutes to let the AI sample your voice
2. It speaks your voice in the language of your choice
3. It also edits the video so that you lip-sync

Explanation with demonstration from English to French & German:

https://twitter.com/x/status/1701075571630047525

Sample:

https://twitter.com/x/status/1687129787267973123

This is pretty cool for things like shooting one video & then translating it out worldwide! I'd imagine we'll see it in near real-time video soon, as Zoom already has live captions for meetings, plus live captions translated to different langauges:

https://support.zoom.us/hc/en-us/articles/6643177746829-Viewing-captions-in-another-language

Google has a free Translate app: (available for iOS, Android, and computer)

Google Translate

Google's service, offered free of charge, instantly translates words, phrases, and web pages between English and over 100 other languages.

translate.google.com

Which they translated into a neat system where you can use your wireless earbuds to speak in your native language & then it speaks out the other person's language on your phone's speaker:

https://youtu.be/7ist0_mPSYM

 

[Kaido]

Neat, you can make Magic Eye autostereogram pictures with AI now! (article requires login)

How to Create Your Own Magic Eye 3D Image with AI Art

How I made an autostereogram using Jasper Art

bootcamp.uxdesign.cc

Unlocking AI Magic Eye by using Midjourney and Chat GPT • The Ink Kitchen

In today’s world, artificial intelligence (AI) is revolutionizing creative fields in fascinating ways. One such groundbreaking exploration involves the collaboration between Chat GPT and Midjourney’s extensive AI toolbox. Chat GPT acts like an advanced thesaurus, capable of learning...

inkkitchen.com

Neural Magic Eye

Neat builder site:

Easy Stereogram Builder

www.easystereogrambuilder.com

 

[Kaido]

Getting right up there on the verge of hyper-realism. Exciting yet also HUGELY scary because we live in the age of misinformation:

1. Don't even need Deepfakes anymore, AI can generate realistic video
2. It can clone your voice (see earlier post)
3. It can make you talk in a different language, complete with lipsync (see earlier post)

https://twitter.com/x/status/1701694553353326831

The AI image-to-motion stuff is just starting out. People are already starting to put together fun projects like 100% AI-generated movie trailers. This trailer was made with Midjourney (AI images), Eleven Labs (voice), Runway (image to video), and Capcut: (video editing)

https://twitter.com/x/status/1701569300924903849

Everyone is adopting AI editing to speed up the video editing process:

AI video editing in Premiere Pro | Adobe

Discover how artificial intelligence (AI) is game-changing for film and video creators by making it faster and easier to manage clips and execute perfect edits.

www.adobe.com

Eleven Labs for voice is REALLY amazing:

1. Speech synthesis (realistic text to speech)
2. Voice library (tweak & share unique voice profiles)
3. VoiceLab (clone your voice from 60 seconds of audio from you)

ElevenLabs - Generative AI Text to Speech & Voice Cloning

elevenlabs.io

This technology has already been used for nefarious purposes...again, scary applications of the technology:

https://www.cnn.com/2023/04/29/us/ai-scam-calls-kidnapping-cec/index.html

 

[Kaido]

RIP security

https://www.tiktok.com/video/7260772169972714798

 

[Drach]

Hank Williams singing Straight out of Compton.

 

[[DHT]Osiris]

RIP security

https://www.tiktok.com/video/7260772169972714798

While a fun demo, at this point at least it still depends on previous exploits having been developed, though this would make things like SQL injections trivial I imagine.

I'd also be more impressed if he had tried to use the exploit and it worked. Chatgpt never really says 'i don't know how to do that', it's happy to confidently give you a wrong answer.

 

[A///]

While a fun demo, at this point at least it still depends on previous exploits having been developed, though this would make things like SQL injections trivial I imagine.

I'd also be more impressed if he had tried to use the exploit and it worked. Chatgpt never really says 'i don't know how to do that', it's happy to confidently give you a wrong answer.

There are far pressing matters of national security when it comes to AI. Have you seen some of the early attempts of replacing voice and speech using the same patterns in another language by way of a third party voice? That's going to be an nightmare in 10 years or less once the mild robotic and twitchiness goes away. The world is in for a world of hurt sooner or later. This is one of the dangers of AI.

 

[[DHT]Osiris]

There are far pressing matters of national security when it comes to AI. Have you seen some of the early attempts of replacing voice and speech using the same patterns in another language by way of a third party voice? That's going to be an nightmare in 10 years or less once the mild robotic and twitchiness goes away. The world is in for a world of hurt sooner or later. This is one of the dangers of AI.

Yeah, deepfakes were bad enough, but just generating content whole-cloth? Most of our accepted reality becomes entirely subjective at that point, you only really know something happened if you were there.

 

[A///]

Yeah, deepfakes were bad enough, but just generating content whole-cloth? Most of our accepted reality becomes entirely subjective at that point, you only really know something happened if you were there.

The deepfakes were never that good. It may be due to the tech at the time vs now. This new stuff while it can be picked out if you're careful is convincing. Give it a few more years and better tech + tech available second hand from DC and it's a recipe for disaster.

We used to be able to tell when an image was edited due to underlying data that could be pulled but new methods were able to hide those changes away. As a youngin i yearned for the future and now that I'm old and looking forward to retirement I roll my eyes at how horrible this world will be in the years to come.

 

[Kaido]

The deepfakes were never that good.

You mean Tom Cruise DOESN'T do industrial cleaning??

 

[Kaido]

3D upscaling is now (or rather, will be Real Soon Now), pushbutton easy! Like...stick a 2D picture in (ex. AI-generated from Midjourney in this case). Click Upscale to 3D. Then...magically...drag the mouse around to change the angle, perspective, and zoom:

https://twitter.com/x/status/1702721761790357733

Very early in the game but still, NEAT!

https://twitter.com/x/status/1702046625353396273

 

[JEDI]