4/23 Summary of Creative Work within this month

Note：

We discuss an AI software called Stable diffusion-Comfyui, which, unlike Midjourney, is open source, making it more controllable and offering a richer ecosystem. However, it is more difficult for beginners to use.

Why AI? Deconstructing the Traditional Animation Process

1.     Traditional animation processes are lengthy. For example, the workflow for traditional 3D animation involves modelling, texturing, effects simulation, animating, lighting, rendering, and post-production adjustments(compositing). This process can take months or even years for an independent artist to produce just a few seconds of animation. AI greatly enhances work efficiency. At its current stage, AI can quickly generate large amounts of animations through text-to-image and image-to-image methods, significantly boosting productivity by laying off those complex procedures. This is crucial for independent artists as it reduces the costs of communication and cooperation, minimizing the technical aspects and emphasizing creativity.

2.     Traditional, stylized animations are costly and technically limiting like Disney or Pixar. AI brings transformative changes in this area by lowering the barriers to creating personalized, stylized animations, thanks to technologies like IPadapter and Lora. Stylization is a key element in experimental animations.

Figure 1. Ai generated animation using my custom woodcut print Lora

3.     Experimental animations generated by AI offer surprises and limitless potential, as seen in the random combinations of characters and color blocks in the short animation below

Figure 2. "Zao God Animation" was created by an AI that used my custom Lora

Figure 3. Zao God Animation

4.     AI, as a deconstruction of traditional animation processes, is highly innovative. It represents an era that breaks from the old and attempts to establish a new order. AI is already beginning to replace parts of the traditional 3D process. Therefore, in this period of transition from old to new, most people are still exploring， like the "hundred flowers bloom in a garden. This is essentially a large-scale social experiment where everyone is trying to use technology to explore and establish their own workflow. For more details, see the Birdman workflow：详见鸟人工作流

What is my current practical idea? And how AI is related to my research topic.

In summary, AI represents a disruptive experiment in traditional animation processes. My research interest can be summarized in one sentence: Using modern techniques to inherit and innovate traditional woodcut prints and expressing Chinese aesthetics through experimental animations.

Innovations in Woodcut Prints？

 -I will visually extract features from Chinese woodcut prints, transforming them from static to dynamic forms.

• AI is a crucial tool for rapid iteration, exploration, and experimentation, as traditional 3D methods are too slow. I will continue to use AI along with various 3D tools and printing techniques to create new works.
  

So like what?

Creative+Idea👇： Combine AI with 3D animation workflow to quickly generate original short animation, followed by post-processing in After Effects or Photoshop, printing, and machine colouring (see video below).

Demo:

- Dataset 1: Training Dataset

- Video 1: Rough Output generated by Comfyui

- Video 2: Rough Final video after compositing

future work that I will do:

详见purz工作流….

Mixed Media Animation

machine colouring

What I‘ve done so far within this month？

• I've conducted a series of experiments and produced several works, including over 1000 testing images, 10+ testing videos, and developed 2 custom GPTs for Lora captioning.
  

video Outputs

My GPTs

• I've familiarized myself with various basic nodes and methods in ComfyUi and tackled the core technical challenge of Lora.

• I've gained a comprehensive understanding of the V2V, T2V, and I2V workflow and applied this knowledge in various practical experiments.

• I've participated in two exhibitions and started keeping daily notes.

What are my reflections and Problems？

Reflection & Finding:

• In this era of transition, traditional 3D processes still have irreplaceable elements. AI has not yet achieved the level of detail required for specific materials and custom requests. we can say that traditional 3D animation techniques complement AI, which in turn is a beneficial supplement to traditional animation

• Currently, AI tends to produce better results for topics like real people, abstract fluids, celebrities, original drawings, and real objects. However, it does not perform as well with custom effects such as woodblock prints. One reason is that the existing large models are trained primarily on current art and real-life images, which does not translate well to niche or endangered styles. This is why I spent nearly a month using Lora to address this issue(see the contrast video below).
  

• AI is still not at the stage where it can produce perfect, commercial-quality outputs. Common issues include multiple character fingers and perspective errors. Therefore, manual adjustments are still necessary to achieve better results. Moreover, AI is not particularly good at creating long videos or exaggerated character animations, so it's important to combine it with other techniques

Problems：

• AI has blurred the boundaries of creation. It acts more like an assistant than a pure digital tool like Photoshop. Thus, it raises the question: Is the work created solely by the artist, or is it a joint creation with AI?

• While attempting to use Riso to mimic the effects of printmaking, current technology and equipment limitations mean colours are not accurately reproduced. (Possible or May lead me to explore other alternative printing methods?).

• My current research subject, Tantou woodblock prints, has too few samples available. After inquiries and research both offline and online, I've found that there are only a few dozen old woodblock prints available. This small number of samples is insufficient to train a robust AI model. Given the lack of intellectual property concepts in ancient China, ancient woodblock prints from different production areas, like Foshan City and Sichuan Province, show interrelated styles and character designs. (Possible leads me to consider broadening my research focus, perhaps to include ancient Chinese prints from before 1840?).