Category: Sessions with Serra_Term 3

After selecting the models, proceed with rendering tests in Unreal Engine to ensure compatibility and visual quality. Always prepare a Plan B in case the models from Maya cannot be imported correctly, such as alternative formats or backup models.

When creating animations, consider the complexity of cloth simulations during post-production. Ensure that limbs and the body have enough space to accommodate these simulations to avoid collisions and clipping issues. Test the cloth behavior early in the process to anticipate and solve potential problems.

During rendering, plan the style of the final video early. Define the visual tone, color grading, and overall aesthetic in advance. Finding suitable music for the rendered video post-production can be very challenging, so having a dynamic storyboard from the start is crucial. This will help in aligning the visual and auditory elements seamlessly.

For aesthetic references, aim for a streamlined selection. Focus on a few key references that embody the desired look and feel. Too many varied references can result in a final product lacking a cohesive style. Create a mood board to visually guide the project and ensure consistency throughout the production process.

This semester has been a period of extensive learning for me. I started with Vicon motion capture, Touch Designer, and Resolume Arena. These foundational skills have opened up new possibilities for creating and showcasing my work.

Delving into Vicon motion capture, I learned the fundamentals of capturing and analyzing motion data, which allowed me to explore the intricacies of human movement. This understanding became a cornerstone for creating realistic animations and simulations in my projects.

In Touch Designer, I discovered the power of visual programming. I worked on creating interactive media and real-time visual effects, pushing the boundaries of visual storytelling. This platform enabled me to design immersive experiences, blending art and technology seamlessly.

My journey with Resolume Arena was equally transformative. I learned to manipulate visual content in real-time, perfecting my skills in live video mixing and projection mapping. This software allowed me to bring dynamic visuals to live performances, enhancing the audience’s experience.

Collaborating with peers on various projects further enriched my learning experience. Working in a team setting provided different perspectives and approaches to problem-solving, which enhanced my adaptability and integration of feedback. This collaboration honed my communication and project management skills, essential for executing complex, multidisciplinary projects.

Attending workshops and guest lectures introduced me to the latest trends and innovations in the field. These sessions offered valuable insights into real-world applications and cutting-edge research, inspiring me to think about how I could apply my newfound knowledge in practical settings. The exposure to industry professionals and their experiences provided a roadmap for navigating my future career in this rapidly evolving landscape.

In addition to technical skills, I delved into the theoretical aspects of digital media, studying design principles, user experience, and the psychological impact of visual storytelling. This holistic approach ensured that my creations were technically sound, meaningful, and engaging to the audience.

By the end of the semester, I had compiled a diverse portfolio of projects demonstrating my proficiency in these new tools and techniques. Each project showcased my growth, creativity, problem-solving abilities, and the capacity to convey compelling narratives through digital media.

In conclusion, this semester has been a transformative period of learning and growth. From mastering Vicon motion capture, Touch Designer, and Resolume Arena to exploring collaborative and theoretical aspects of digital media, I have gained a comprehensive skill set that has broadened my creative and professional horizons. I am excited to continue building on this foundation, exploring new technologies, and pushing the boundaries of what I can create and achieve in the future.

This is the final composition vedio.

It’s time to dive into Unreal!

I had to scrap one of the scenes because I encountered some compilation errors when rendering it.

Here’s my workflow: I imported the naked model into Marvelous Designer (MD) using the Alembic Cache format, then performed cloth simulation. After that, I exported the cloth separately and re-imported it back into Maya(Alembic Cache). In Maya, I combined the character with the cloth Alembic Cache file, assigned materials, and exported it as a single file (Alembic Cache) to bring into Unreal. And build the texture in Unreal.

These are some sequence in Unreal with the camera movements.

Editing Techniques for Seamless Shot Transitions

To facilitate smoother transitions between shots during post-production, Serra suggested several strategies:

1. Switching Between Similar Scenes:

• Transition between shots that share similar settings or backgrounds to maintain visual continuity.

1. Utilizing Camera Rotations:

• Make use of moments when the camera is rotating to seamlessly switch to a different shot, creating a dynamic flow.

1. Matching Body or Limb Movements:

• Transition using similar body or limb movements to ensure fluid motion and consistency between shots.

Shot Composition and Techniques

• Shot Variety:
• Incorporate both wide shots and close-ups to provide a diverse perspective and enhance storytelling.
• Use of Rotating Shots:
• Employ numerous rotating shots to achieve a dynamic and engaging visual style, enhancing the narrative’s impact.

Post-Rendering Workflow

After using Movie Render Queue to render the sequence frames, I utilize CapCut to merge the frames and add special effects. CapCut provides a user-friendly interface and powerful tools for combining sequence frames, allowing for smooth transitions and dynamic effects. This post-rendering workflow helps in refining the visual output and enhancing the overall quality of the final video.

The Output section of Resolume is responsible for sending video content to various display devices. It allows you to select output devices, set resolution, perform Output Mapping, and provides advanced options like edge blending and geometric correction. You can preview the output effects and it supports Spout and NDI technologies for real-time video signal transmission with other software and devices. Additionally, you can save and load output presets, making it adaptable to various complex display requirements.

The Output Transformation can change the different slides in to different screen, so the output can be easily show high quality in the combine displays .

The size on output can be change so can be cast on a specific object.

Then we tried to put our own vedio on to different monitor.

The software we use is called Resolume Arena

Resolume Arena is a powerful real-time video mixing and VJ software designed for live performances and multimedia displays. It offers a flexible interface and rich features, including multi-layer video mixing, audio-reactive visuals, numerous effects and transitions, advanced Output Mapping, multi-screen output, and support for Spout and NDI for real-time video signal transmission with other software and devices. Additionally, it provides convenient media management tools, making it ideal for concerts, art installations, and other multimedia events.

And it can be connected with the keyboard

can drag and drop it in the same time or it can separate drag and drop in different layer and column, differnt layer can show the mix output by order

The details can be change at clip part. The effect can be drag and drop into layer or the clips, the front one will effect all layers and the back one only effect the specific layer.

We can change the shortcut by ourselves. And it can be selected to control all layers or just one clip by change the Shortcuts target.

This is the Live output I recorded.

When attempting to import the initial rig (character skeleton) into Unreal Engine, we encountered difficulties, particularly with texture mapping. To address this issue, I tried using Metahuman, a built-in character creation tool in Unreal Engine, but faced real-time facial capture issues when redirecting it to Unreal Engine. After two weeks of effort with no progress, we decided to revert to using the initial model.

Subsequently, I took steps to improve the initial model. I cleaned up the model’s history and re-mapped the UV textures，then used Substance Painter software to create new textures and re-bound the character’s skeleton using the Advance Skeleton tool.

Process in Substance Painter

Process in Marvlous Designer

After completing animation work in Maya, I conducted cloth testing and simulation using Marvelous Designer. Marvelous Designer is a professional cloth simulation software that can simulate the movement and deformation of cloth, adding a more realistic clothing effect to characters.

First, I created a circle. After adjusting its size, I edited the edges of the circle using noise. Then, I used the level and composite tools to modify the inside of the circle.

Next, I imported a photo and connected it, changing the base circle’s color. I added a transform and adjusted the rotation.

Finally, I added an edge node to enhance the edge variety, then adjusted the color and noise details. I used the moviefileout to export the final result.