Real-time animated digital doubles at Eisko

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
 24
 
  1. Introduction EISKO : Digital Doubles production for entertainment Digital double = photoreal & accurate 3D representation of personalities appearance &…
Related documents
Share
Transcript
  • 1. Introduction EISKO : Digital Doubles production for entertainment Digital double = photoreal & accurate 3D representation of personalities appearance & performance
  • 2. EISKO Process analysis and reconstruction of HD acquisitions Appearance Separation between: Morphology & Materials processing Performance Separation between: Expressions & Dynamics processing
  • 3. EISKO Process analysis and reconstruction of HD acquisitions Measures acquisition > Data analysis > Model reconstruction Appearance Separation between: Morphology & Materials processing Performance Separation between: Expressions & Dynamics processing
  • 4. EISKO Technology Based on 2 technological assets:
  • 5. EISKO Technology Based on 2 technological assets: • Proprietary capture system transportable, fast, accurate simultaneous geometric & textures acquisitions of every expression
  • 6. EISKO Technology Based on 2 technological assets: • Proprietary capture system transportable, fast, accurate simultaneous geometric & textures acquisitions of every expression • Proprietary reconstruction technology - Data > Model registration - Information transfer/retargeting - Characterization of adapted expressions spaces
  • 7. EISKO Process 3 production deliveries:
  • 8. EISKO Process 3 production deliveries: › Appearance Digital Double morphology and material reconstruction
  • 9. EISKO Process 3 production deliveries: › Appearance Digital Double morphology and material reconstruction › Animatable Digital Double expressions reconstruction
  • 10. EISKO Process 3 production deliveries: › Appearance Digital Double morphology and material reconstruction › Animatable Digital Double expressions reconstruction › Animated Digital Double performance retargeting
  • 11. 1. Appearance digital double Highly detailed geometric and physically-based textures capture & reconstruction Shading set-up ready–to-be rendered in Unity 5
  • 12. Traditional capture systems Traditional approaches:
  • 13. Traditional capture systems Traditional approaches: › Contact measurements
  • 14. Traditional capture systems Traditional approaches: › Contact measurements › Manual scanner
  • 15. Traditional capture systems Traditional approaches: › Contact measurements › Manual scanner › Structured-light acquisition
  • 16. Traditional capture systems Traditional approaches: › Contact measurements › Manual scanner › Structured-light acquisition › Photogrammetric acquisition
  • 17. Photogrammetry Pros: › Easy to build
  • 18. Photogrammetry Pros: › Easy to build › Rapid acquisition
  • 19. Photogrammetry Pros: › Easy to build › Rapid acquisition Cons: › Noisy and partial geometry
  • 20. Photogrammetry Pros: › Easy to build › Rapid acquisition Cons: › Noisy and partial geometry › Image color only
  • 21. EISKO Capture Eisko’s approach: › Fully transportable Easy to set-up (2h) › Simultaneous geometry+material capture of any expression › 360 Highly-detailed geometry › Physically-based material components
  • 22. EISKO Geometry acquisition
  • 23. EISKO Geometry reconstruction Model is reconstructed from data, through dense correspondence
  • 24. EISKO Geometry reconstruction Model is reconstructed from data, through dense correspondence
  • 25. EISKO Geometry reconstruction Model is reconstructed from data, through dense correspondence
  • 26. EISKO Geometry reconstruction Reconstruction based on Customer’s topology
  • 27. EISKO Geometry reconstruction Reconstruction based on Customer’s topology
  • 28. EISKO Material acquisition
  • 29. EISKO Material acquisition
  • 30. EISKO Material acquisition
  • 31. EISKO Material acquisition
  • 32. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 33. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 34. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 35. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 36. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 37. EISKO Material reconstruction Reconstruction based on customer’s UVs
  • 38. Physically-based shading in Unity 5
  • 39. Characterization of extrems expressions
  • 40. Appearance Digital Double in Unity 5
  • 41. 2. Animatable digital double Highly detailed expressions capture & reconstruction Animation rig ready-to-be animated in Unity 5
  • 42. Facial deformations Two problematics / Two different approaches
  • 43. Facial deformations Two problematics / Two different approaches Fast (generic) Targetshapes automatic generation
  • 44. Facial deformations Two problematics / Two different approaches Fast (generic) Accurate (specific) Targetshapes automatic generation Targetshapes reconstructed from capture
  • 45. Facial deformations Two problematics / Two different approaches Fast (generic) Accurate (specific) Targetshapes automatic generation Targetshapes reconstructed from capture
  • 46. Targetshapes automatic generation: Automatic retargeting with morphologic adaptation
  • 47. Facial deformations Two problematics / Two different approaches Fast (generic) Accurate (specific) Targetshapes automatic generation Targetshapes reconstructed from capture
  • 48. Targetshapes reconstructed from capture: Capture protocol:
  • 49. Targetshapes reconstructed from capture: Capture protocol: › facial motions
  • 50. Targetshapes reconstructed from capture: Capture protocol: › facial motions › emotionnal expressions
  • 51. Targetshapes reconstructed from capture: Capture protocol: › facial motions › emotionnal expressions › visemes
  • 52. Targetshapes reconstructed from capture: Capture of a full range of deformations specific to the talent linked to facial motions, visemes, emotional expressions
  • 53. Targetshapes reconstructed from capture:
  • 54. Targetshapes reconstructed from capture:
  • 55. Targetshapes reconstructed from capture:
  • 56. Targetshapes reconstructed from capture:
  • 57. Targetshapes reconstructed from capture:
  • 58. Targetshapes reconstructed from capture:
  • 59. Reconstruction of specific targetshapes Targetshapes reconstruction through rigid & non-rigid matching, and facial action units localization
  • 60. Reconstruction of specific targetshapes Targetshapes reconstruction through rigid & non-rigid matching, and facial action units localization
  • 61. Reconstruction of specific targetshapes Targetshapes reconstruction through rigid & non-rigid matching, and facial action units localization
  • 62. Stress ratio characterization Displacement and normal details are encoded through color vertices Regarding extrems deformations
  • 63. Stress ratio characterization Displacement and normal details are encoded through color vertices Regarding extrems deformations
  • 64. Stress ratio characterization Displacement and normal details are encoded through color vertices Regarding extrems deformations
  • 65. Stress ratio characterization Displacement and normal details are encoded through color vertices Regarding extrems deformations Green vertices : Uncompress ratio Red vertices : compress deformations
  • 66. Facial animation rig
  • 67. 3. Animated digital double High-fidelity performance retargeting Management of animation performances in Unity 5
  • 68. High-fidelity performance retargeting Characterization of the pose in the expression space drives the animation
  • 69. High-fidelity performance retargeting Characterization of the pose in the expression space drives the animation
  • 70. High-fidelity performance retargeting Characterization of the pose in the expression space drives the animation
  • 71. High-fidelity performance retargeting Characterization of the pose in the expression space drives Facial Action Units (FAU) animation
  • 72. Body animation
  • 73. Facial animation Expression space driven animation: With all 84 FAU
  • 74. Control on fidelity and performance Reducing expression space dimensions > higher performances, while maintening fidelity FAU are importance-sorted, allowing easy optimisation : Reduced to 10 FAU Reduced to 5 FAU
  • 75. Real-time animation in Unity 5 Proprietary components overide standard blendshapes management
  • 76. Conclusion & perspectives General overview of our real-time digital doubles offer:
  • 77. Conclusion & perspectives General overview of our real-time digital doubles offer: › HD Capture approach delivering accurate geometry+texture+deformations ground-truth › Reconstruction based on customer’s topology through data/model dense correspondence › Control over fidelity and performances through Expression space animation › Physically-based shading and Real-time animation in Unity 5, or any real-time engine
  • 78. Conclusion & perspectives General overview of our real-time digital doubles offer: › HD Capture approach delivering accurate geometry+texture+deformations ground-truth › Reconstruction based on customer’s topology through data/model dense correspondence › Control over fidelity and performances through Expression space animation › Physically-based shading and Real-time animation in Unity 5, or any real-time engine Multiple alternatives and combinations depending on your project !
  • 79. Contact us! www.eisko.com
  • 80. Thanks! Talent Tatiana Seguin EISKO Cédric Guiard Gilles Gambier Alexis Caillet Florian Croquet Pierre Lelievre Romain Lopez Benjamin Vedrenne Remi Deletrain – Fabien Goeau - Bruno Etchepare UNITY support David Miranda Mathieu Muller Partners OSEO / BPI Region Ile-de-France Ville de Paris Cap Digital
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks