- Facial rendering,
- Facial and speech animation,
- Hair modelling,
- Body rendering and modelling,
- Body mechanics and animation,
- Clothes modelling, and
- Body physiological model.
Facial renderingIt wasn't feasible to include any decent images of the current state-of-the-art in facial rendering in the book, so here are some of our favourite examples. Note that a key distinction is between those that are rendered in non-real time (e.g. for movies) and those that are rendered in real-time (for games and virtual worlds). Our real interest is very much in the later, which is probably a few years behind the best of what is being done in non-real time.
Here are some examples of state-of-the-art facial renders, and a few historic classics.
Jue, Thadeus and the crew from Final Flight of the Osiris - the CGI episode in Animatrix. Again film CGI but state of the art in 2003.
Nice contemporary face render from http://www.chrisj.com.au/
- https://www.youtube.com/watch?v=1pfqCQQIbUM and https://www.youtube.com/watch?v=EQ8FywbxYf8
- Umajin system - https://www.youtube.com/watch?v=6loN4kkqTVg
- Nice time-lapse video of some of the effort all this takes - https://www.youtube.com/watch?v=Ma17a_M6wYM
Facial and speech animationThe following links show videos of (usually real-time) facial animation and speech animation.
Some really nice real-time facial rendering, animation, lipsync and emotive voice from Soul Machines.
The whole "DeepFake" movement is also of interest, although it is not usually in real time it is often running on quite simple hardware and against almost any target face. Take this example:
Hair modellingThe following links show images and videos of (usually real-time) hair modelling.
Early Second Life "skull-cap"/fixed-prim hair
Mid Second Life Flexi-prim hair
Modern SL Mesh Hair - see also video at https://www.youtube.com/watch?v=Yj1bsNcTQB0 for wind-blown animation effects.
Interesting papers and web sites are:
- Video of Disney research in this area
- Short Khan Academy introduction to the topic, using the red-headed Princess Merida from Disney's Brave as an example
- A 2012 research video - focus on movement not colour
- Bertails, F., Audoly, B., Cani, M. P., Querleux, B., Leroy, F., & Lévêque, J. L. (2006). Super-helices for predicting the dynamics of natural hair. In ACM Transactions on Graphics 25 (3), 1180-1187.
- Bertails, F., Hadap, S., Cani, M. P., Lin, M., Kim, T. Y., Marschner, S., & Kačić-Alesić, Z. (2008). Realistic hair simulation: animation and rendering. In ACM SIGGRAPH 2008 classes. ACM.
Body rendering and modellingThe following links show images and videos of (usually real-time) body rendering and modelling. May be NSFW!
Other examples and interesting links:
- Real-time (but static) rendering in Unity 5 (2015)
- Hobbyist produced game style renders
- Staples, A. (2012). Subsurface Scattering: Skin shaders for Poser 9/Poser Pro. Penultimate Harn website.
Body mechanics and animationThe following links show images and videos of (usually real-time) body mechanics and animation.
- Real-time mo-cap animation - https://www.youtube.com/watch?v=TyCc2186viY
- iClone system - https://www.youtube.com/watch?v=bZ78loEhAmg
- Human - Avatar interaction - https://www.youtube.com/watch?v=Dqy-mVj0R7g
- Hololens avatar animation - https://www.youtube.com/watch?v=f-FSFI0Ta8s
- Stuart, K. (2015) “Photorealism - the future of video game visuals”, Guardian website (12 Feb 2015).
- Visage. (2012) MPEG-4 Face and Body Animation (MPEG-4 FBA) An overview.
Clothes modellingThe following links show images and videos of (usually real-time) clothes modelling and useful articles and papers.
- Great real-time demo of dancer in long dress showing cloth dynamics and response to body movements - https://www.youtube.com/watch?v=KBfxnayIlOY
- Hi-rez GPU accelerated cloth modelling (as particles!) - https://www.youtube.com/watch?v=kCGHXlLR3l8
- Nice basic demo of two different cloth modelling techniques - https://www.youtube.com/watch?v=kLUC7xbuTlU
- Another very nice GPU accelerated demo - https://www.youtube.com/watch?v=upEut4kl0YM
- Well animated cloth in UE4 - https://www.youtube.com/watch?v=BJ3NlyXu570
- 3D anime style animation demo using Blender EEVEE real-time - https://www.youtube.com/watch?v=aikOQjYqQF8
- Aliaga, C., O'Sullivan, C., Gutierrez, D., & Tamstorf, R. (2015). Sackcloth or silk?: the impact of appearance vs dynamics on the perception of animated cloth. In Proceedings of the ACM SIGGRAPH symposium on applied perception 41-46
- Wu, K., & Yuksel, C. (2017). Real-time fiber-level cloth rendering. In Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM.
- Brooke, E (2104) “8 Startups Trying To Help You Find Clothing That Fits” on the Fashionista blog (22 Jul 2014). Available online http://fashionista.com/2014/07/8-tech-startups-tackling-clothing-fit
Body physiological model.The following links show images and videos of (usually real-time) hair modelling.
Interesting papers and web sites are:
- European Virtual Physiological Human (VPH) initiative
- Virtual Physiological Human Initiative
- Kramar, V., Korhonen, M., & Sergeev, Y. (2013). Particularities of visualisation of medical and wellness data through a digital patient avatar. InOpen Innovations Association (FRUCT), 2013 14th Conference of (pp. 45-56). IEEE.
- Lopez, C. (2014). Latest 'Virtual Battle Space' release adds realism to scenarios, avatars. On www.army.mil (linking avatars to human performance)
- UCL (2015). The Virtual Physiological Human.