The Wait
Sep 2016 Team of 5 in 2 Weeks
The Wait is augment reality storytelling experience in Microsoft Hololens. We try to create the feeling when you are waiting for your daughter taking a surgery. It's a 2 weeks project by team of 5 in Entertainment Technology Center (ETC) of Carnegie Mellon University (CMU).
Overview
- Role: Lead Programmer, Visual Effect Designer
- Platform: Unity 3D, Microsoft Hololens
Responsibility
- Using Hololens SDK to combine Unity 3D with Microsoft Hololens
- Implemented all code base for level system
- Implemented part of level programming
- Implemented visual effect
Challenges
Paper Drawing Animation
One of the scene in the game need to have the little girl drawing on a paper, I implemented this effect with customize shader for this part.
void surf(Input IN, inout SurfaceOutput o) { float4 clr = tex2D(_MaskTex, IN.uv_MaskTex); if (clr.r + clr.g + clr.b <= _FadeParam) o.Albedo = tex2D(_MainTex, IN.uv_MainTex).rgb; else o.Albedo = _BackgroundColor; }
Calibration
Calibration is an important part in this augment reality experience, because we were trying to make virtual character interact with real object. I use voice recognition to control the calibration, and put the object root in a fixed position in the real world. After the calibration of virtual world, we put all the props in real world in the correct position according to place holder that shows in hololens.
Clipping
Spatial mapping is not accurate for clipping virtual object, we built a set of virtual object to clipping the virtual character. Modifying the render queue of the material so that the virtual object will be render at first, and write z value to stop rendering of virtual characters.
SubShader { Tags { "RenderType" = "Opaque" "Queue" = "Background+1" } Pass { ColorMask 0 //Offset 50, 100 CGPROGRAM #pragma vertex vert #pragma fragment frag #pragma exclude_renderers d3d11_9x #include "UnityCG.cginc" struct v2f { float4 pos : SV_POSITION; }; v2f vert(appdata_base v) { v2f o; o.pos = mul(UNITY_MATRIX_MVP, v.vertex); return o; } half4 frag(v2f i) : COLOR { return float4(1,1,1,1); } ENDCG } }