Articles | Jie Zhou Art

Working as a DMP/Env-Generalist for COMMERCIAL AND FILM VFX

It's been 4 years since I moved from Los Angeles to Vancouver, marking my transition from the commercial industry to the film industry. During these 4 years, I have worked on numerous film projects at Scanline VFX and Weta FX in Canada. It has been a significant shift for me, working in the film industry compared to the commercial projects. Here, I will specifically outline how I worked on commercial and film projects.

Matte Painting Workflow in the Commercial VFX Pipeline

This is the matte painting workflow I followed in the professional commercial VFX pipeline. The commercial pipeline is fast-paced, requiring shots to be completed within a short timeline, with most common shots needing to be finished in just 2 days. This workflow is applicable to different studios and a wide range of commercial projects.

Start with the Brief

The Producer/VFX supervisor initiates the shot with a briefing meeting with the matte painter. These meetings can be conducted in person or online and serve to provide me with a clear understanding of the project's requirements.

Determining the Matte Painting Workflow

The matte painting workflow is determined based on the specific shot, taking into account factors such as perspective, whether the scene is organic or manmade, lighting, and camera movement. Discussions are held with the CG project lead to ensure seamless integration of matte painting with CG elements.

Common DMP Workflows for Commercials

The workflow must align with the quick turnaround times typical of commercial projects. I often use a 2D-2.5D workflow, occasionally incorporating 3D elements for special cases.

2D: Compositing photo references in Photoshop, delivered as PSD or TIFF files with alpha channels. This approach is suitable for locked shots or simple camera movements and is commonly used due to its speed and cost-effectiveness.

2.5D: Layer preparation in Photoshop with projection in Nuke, resulting in rendered layers with alpha channels. This approach is suitable for shots with camera movement and involves tracking, 3D cameras, 3D models, and rendering.

3D + Photoshop Retouch: Creating a basic 3D shape in Maya and refining details in Photoshop. This method is useful for shots featuring complex manmade structures or interior and architectural elements.

Matte Painting Workflow Software: Utilized software includes Photoshop, Maya, Vray, and Nuke.

Steps from Beginning to End of a Commercial Project

Acquire the Live Action Plate: Carefully inspect the live action plate for matte painting, ensuring the selection of the correct frame. Correct any lens distortion before beginning work.

Reference Research: Gather both conceptual and high-definition references or use references from the kick-off meeting.

1st Pass: Achieve 70-80% progress, creating 2-3 versions for discussion if necessary.

2nd Pass: Make changes based on notes from internal reviews, client feedback, or agency notes.

Continue iterations until everyone is satisfied, organize PSD or EXR layers with clean alpha channels, test in Nuke, and deliver the pre-composited files to the compositors.

Timeline

The 1st pass typically takes half a day or a full day, while a common final matte painting project is usually completed within 1-2 days, with progress updates provided every half or full workday.

Matte Painting Workflow in the Film VFX Pipeline

Compared to commercial projects, which usually involve 2-4 shots requiring matte painting, film projects typically consist of hundreds of shots across multiple sequences. Consequently, film projects are produced by larger teams. Instead of 1-2 matte painters, there are typically 4-8 team members dedicated to matte painting for a single film project within a studio. Due to the extensive number of shots and sequences, as well as the complex technical requirements, a lead is responsible for determining the matte painting workflow, budget, and schedule for production. All team members work on assigned tasks, with their work reviewed by the lead and CG supervisor. The time required for one shot can range from 1-4 weeks, or even longer, depending on internal and directorial feedback.

Common DMP Workflows for Films

In many cases, most shots are completed at 80-100% with full CG. Environment generalists like myself are responsible for creating full CG environments with support from other departments. Buildings and props assets are completed by the modeling department. Plates are prepared with basic cleanup, lens distortion correction, and color grading. Tracking and layout are handled by the layout department. The environment team assembles all elements, including creating assets like procedural terrains and vegetation if necessary. Once all assets are prepared, we perform assembly, vegetation scattering, look development, and render full sequence with lighting passes. We submit updated versions for dailies reviews, incorporate creative notes, conduct final quality checks, and ultimately deliver to the compositing department. Additionally, as digital matte painters, we sometimes use 2D and 2.5D techniques to retouch CG renders or live-action plates. Different companies may employ varying matte painting workflows, even for the same film. For example, I worked on the Marvel movie 'Shang-Chi and the Legend of the Ten Rings' in 2022 at Scanline VFX, where I spent nearly a year using 2D and 2.5D workflows to enhance CG renders with details from photo references. Conversely, at Weta FX, everything was accomplished using CG. Regardless of the workflow used, the final movie achieved great success and received an Oscar nomination for Best Visual Effects.

Software Used in the Film Industry

Regarding software used in the role of an environment generalist, the situation is complex. Unlike other departments, such as compositing, where Nuke is the primary tool for completing 95% of the work, environment generalists must be proficient in modeling, texturing, lighting, rendering, and compositing. We use Maya, Houdini, and Geae for hard surface modeling and creating procedural terrains. SpeedTree is used for vegetation creation. For texturing, tools like Substance Painter, Mari, and occasionally Clarisse for procedural texturing are used. In terms of assembly and look development, we previously relied on Clarisse but have recently transitioned to Houdini Solaris. New software tools emerge every 1-2 years, each introducing more advanced techniques to enhance workflow efficiency. Software such as Blender and Unreal Engine has gained popularity among artists in recent years. As environment generalists, we must continually learn and adapt to stay current and maintain our proficiency.

In Conclusion

In the film industry, artists have the opportunity to focus more on their specific roles with longer timelines compared to the quick turnover demands of commercial projects. The field has become increasingly technical, and the sophisticated pipeline ensures the quality of the final result. Conversely, the commercial pipeline often requires working in small teams and taking responsibility for all matte painting shots in a project, allowing for the full utilization of one's abilities and a greater sense of accomplishment.

This insight is based on my personal experience and is intended for all professionals interested in digital matte painting.

Jie Zhou

September 2023

Matte painting Photo references Resources

Since I start doing the matte painting from 2012, a big part of the matte painting job is done by compositing photo references.

It's a fast and cheap solution to achieve a realistic look in a fast CG pipeline.

In my experience, 80% matte painting work on the fast commercial pipeline is done by photo references.

But It always needs to spend time doing research in the right perspective, lighting, quality, style... Picking up a photo is based on a lot of experience for judging. Good photo reference resource helps a lot for a better final look.

Today I'm sharing my matte painting photo references resources. Hope this can help matte painters and very appreciate if you can share some resources with me too.

Most of the time I need to pay for it, but some free resources are good too. For those images find online for free, we need always pay attention to the copyright. In case we get in trouble from copyright, we should always modify or use part of the photo form those free photos.

1.EnvironmentTextures.com - I paid as you can see the price below. I can feel safe about the quality, and I love their snow mountains and sky a lot. They have a set of the photo including different angles, which is good for a wide scene. But like most of the photo reference websites, it's hard to find what I need if it's not a common thing.

Screen Shot 2018-09-11 at 8.24.29 PM.png

2.https://www.textures.com/- It used to call cgtextures.com. It changed to textures.com in the last few years. Almost everyone does CG knows it, most CG production company owns an paid account for artists to use. It's safe for qualities, and they have 10k resolution pictures. Just a lot of pictures are popular enough so I will try to avoid to use the ones too popular. It's good for both 2D/3D use.

3. Google image - everyone knows It's powerful photo reference research engine, type in what you need and set up the size, a lot of things will come up to you, but just double check the quality or sometimes it's just for internal concept stage, need to modify it based on direction later. I met the situation 1-2 times, the producer came to double check with me for the resources of my photo reference, just in case we get in trouble later. (Example: a Christmas decoration on the window, if it's someone's copyright for designed products…) In my experience, it's safe to use so far, but always need to be careful.

4.mattepaint.com - A new website just came out, and very potential for developing. It might be the best matte painting photo references resource later. I like the function I can choose the perspective when doing the research. All photos are the raw format with a large range of adjusting, which is very professional. One thing amazing is the development team is very friendly to their customers, they keep in touch with me asking feedbacks for user experiences, and really changed based on highly demanded requests. Very appreciate they’re making matte painter’s life easier.

5. Onset photo from jobs. My favorite resource. Each job has a photographer(producer/VFX supervisor) on site taking pictures when they're shooting. That's always the best resource. But I can only get the support form a professional team, not for personal works.

There are some other resources like Shutterstock/ Dreamstime...but I didn't try it. Every time when I was doing my job I just want to grab something as soon as possible. When I find something good in these two but always stopped by register and payment things. Also, the price is not that fare

Those are based on my experiences, hope this can help or plz let me know if you have more ideas.

Jie Zhou

Sep.2018

Beyond Human Challenge: Matte Painting Winners Interview

Original :

https://magazine.artstation.com/2017/11/human-challenge-matte-painting-winners-interview/

In the Matte Painting portion of the challenge, participants had to create one master digital matte painting of an environment where the next evolution of humanity lives and then render it through a slight camera move, using camera projection techniques.

Browse all Matte Painting entries for the Beyond Human Challenge.

The winners Jie Zhou, Jacek Pilarski and Maciej Biniek share their experience with the challenge and process for how they created the photorealistic result.

Sarra: How did you interpret the theme and how did this reflect in the scene you created?

Jie: I think there is a lot of possibility of "Beyond Human" for starting with the concept art. There are some directions I tried. Like the frozen landscape, manmade homeland on the ocean, even we moved to Mars...I tried some sketch for them. By looking at those sketches, some of them are interesting but too crazy for production, some have a good story but not that good for telling it by a piece of painting. At the end, I picked up the cityscape one, which is the most potential for me to develop and being able to finish on time.

In the working progress creating this cityscape matte painting, I switch myself from different roles. I need to think about the painting from art direction perspective, how to design the lighting, how to make the compositing looks better...Also, I need to look back to think about how it makes sense for the topic. I think about the current situation of human being, and how we can improve more based on what we have. For this piece, everything I made is based on our current environment but more futuristic which you could imagine it will happen and makes sense for decades in the near future.

Sarra: As a first time or returning challenger, what was your strategy for the challenge?

Jie: This is my first-time attending matte painting challenge at Art Station. When I saw this is a challenge specific to matte painting, I just came up the way I did my full-time job. I think my strategy is first knowing how matte painting works, and second listen to feedbacks. From my work experience, matte painting is a faster and cheaper way to make things happen by using 2D or 2.5D solution than CG, which works for a fast VFX pipeline. Most of the daily work as a matte painter is creating things based on a live-action plate. Nothing too fancy but everything need to be sitting on the plate properly and update with critiques. Also, listen to feedback is very important. In the progress of challenge, I post my updates and I got some reply for feedback, then I listen and update. I think that's just like part of my job. At work, we will get internal notes, also notes from the agency or director, I need to always be patient and came up the way how to update it properly. In the challenge, there are no clients so I look at the replies and ask around for feedback from my friends, sometimes they give me inspiration from a totally different angle, it's always interesting to know what people are thinking when they looking at my painting. So, I think my strategy is knowing how matte painting works, and ask for many feedbacks and put good thoughts into my painting.

Sarra: Were there any other submissions you were following?

Jie: Hu Te, and Bin Zhu, they're my friends, so I following their updates and talk about our work sometimes. Hu Te makes art by coding, which is incredible and very talented. Bin Zhu and I used to work for graduation thesis together, he has a solid knowledge of modeling always have things done properly. I'm very happy to find I have friends attending the challenge together, it's a happy experience.

Sarra: What do you think is the most difficult part of the matte painting challenge?

Jie: Totally agree it's the beginning, at the time when I need to decide what to paint with a certain topic. As I mentioned before, I tried a lot of possibilities, and I failed a lot too. That's really a hard time, need to decide even to give up or keep on trying new ideas. Also, it's hard when I was very busy at work near the challenge deadline. At the time I almost give up because it's too busy to keep on doing the challenge by looking at my schedule. But I got encouraged by seeing people's replies and my friends, and I really enjoyed the progress doing this piece of personal work, so I decide whatever how busy I want to finish it. I use my sleep time to finish the piece at the end, which I don't recommend you to do it too much. Too many good excuses stopped me these years doing some personal work, so I very appreciate the chance push me finished a piece of personal work.

Sarra: What is your advice for future challenges?

Jie: I don't have too much experience with challenges, but it's a really good motivation to do personal work. It's also a practice for creating art by sticking to a certain topic. I think to enjoy the progress is the most important thing for doing a piece of work, that's the motivation for me to keep happy doing my job for many years. Don't think about the happy ending or not, just enjoy doing it, and it always worth by thinking back each step we did.

Author

Sierra Mon

The Editor of ArtStation Magazine.

Matte painting Workflow in Commercial VFX pipeline

This is the matte painting workflow I worked on professional commercial VFX pipeline. The commercial pipeline is fast, needs to do shots in a short timeline, the most common shot should be done in 2 days.

This workflow works for different studios, hundreds of jobs.

Producer/VFX supervisor kick off the shot with matte painter

Determine the matte painting workflow based on the specific shot (perspective, organic or manmade scene, lighting, camera movement), discuss with CG project lead how matte painting is going to work with the CG parts.

Different MP production workflows：

2D - Compositing photo references in Photoshop, deliver as a PSD file or tiff files with alpha. The 2D solution is good for a locked shot, or simple camera movement. Most matte painting jobs are done with this solution. It's fast and cheap but has a lot of limits.

2.5D - Have layers ready in Photoshop and projection in Nuke, deliver as rendered layers with alpha. 2.5D is good for shots which have camera movement. It involves tracking, 3D camera, 3D model, render.

3D+photoshop retouch - render a basic shape for an 80% look in Maya and retouch details in Photoshop. This solution helps shots have some complicate manmade structures, interior, architectures.

Full 3D - Sketch in Photoshop as a guide, use Maya for modeling, texturing, lighting, rendering. This is good for a complicated scene.

Matte Painting workflow software: Photoshop/Maya/Vray/Nuke/Vue/Worldmachine

Live Action Plate - Double check the live action plate for matte painting. Find the right frame to work on. Undistort the lens distortion before starting.

references research- Need both concept references and HD references, or pull up the references from kick-off meeting.

1st pass - 70-80% progress, 2-3 versions for discussion if needed.

2nd pass - make changed based on notes, (internal notes, client note, agency notes)

Until everyone is happy, organize layers with clean alpha, test in nuke, deliver to compositors with a pre-comp.

Time: 1st pass used to be half or one day. a Common final matte painting usually finished in 1-2 days. Showing progress each half or one work day.

This is based on my personal experience, sharing for all crews who are interested in matte painting works.

Jie Zhou

Sep.2018

Writer Strike and the current Visual Effects Industry situations

Background:

On May 2, 2023, the Writers Guild of America (WGA) initiated a strike that has been ongoing for several months.
The strike's primary reasons revolve around disputes between the WGA and the Alliance of Motion Picture and Television Producers (AMPTP), including issues related to streaming residuals and the use of artificial intelligence in content creation.

Demands:

The WGA seeks to secure a larger share of streaming residuals, as they believe that the AMPTP's proposals would decrease writers' income.
The WGA demands that artificial intelligence (such as ChatGPT) be used only as an auxiliary tool rather than a complete replacement for writers.
The WGA has also proposed requirements for "staffing levels" and "hiring periods" and suggested providing separate pension and healthcare funds for each member of the writing team.

Current Status:

The strike has had far-reaching effects, leading some production companies to take emergency measures, such as pay cuts, to address financial difficulties. For example, visual effects company DNEG implemented pay cuts to cope with industry challenges.
This demonstrates that the strike has not only impacted writers but has also had a negative effect on the entire Hollywood production ecosystem.
Negotiations have been at an impasse, but during a meeting on September 20, 2023, both sides agreed to resume talks in hopes of making progress.

Impact of the Strike:

Due to the Hollywood strike, film and television production has been disrupted, leading to economic challenges for many production companies. For instance, visual effects companies like ILM have closed their Singapore branches, and DNEG has implemented pay cuts to navigate the industry's financial challenges. These pay cuts may involve reducing employee salaries or downsizing staff to reduce costs and ensure business sustainability. This illustrates the widespread impact of the strike on Hollywood and its associated industries, affecting not only writers but also professionals in the visual effects and other sectors closely tied to film and television production.

My Perspective and Advice:

The visual effects industry has experienced a turbulent period in recent years, with the writer's strike being the latest challenge.

During the COVID-19 pandemic, the industry witnessed a brief slowdown, but it later entered a period of unprecedented busy work. From May 2023 onwards, many visual effects companies have experienced a significant slowdown in their work. It's important to note that most companies currently have no projects lined up beyond the end of this year. This situation is reminiscent of the industry's "winter" during the pandemic. The visual effects industry is expected to face another period of slowdown in the first half of 2024, with a resurgence of work anticipated in the latter half of 2024, along with increased involvement of AI technologies in content creation.

For those aspiring to enter the industry or professionals within it:

Keep learning new technologies, including AI, and consider them as tools to enhance your creative efficiency.
Utilize the industry slowdown as an opportunity to rest, improve your skills, and prepare for the next wave of busy work expected in 2024-2025.

There's no need to be discouraged by the current challenges in the North American film and television industry. Every industry experiences ups and downs. The key is to prepare your portfolio and resume so that when opportunities arise during the next busy period, you are well-prepared to seize them.

Stay inspired!

Jie Zhou

09/22/2023