Tips and Tricks for 3D Interfaces on Mobile Devices
Mauricio “Maltron” Leal
Developer Community Maven Sun Microsystems Learning Services – Technology Group
TS-6258
TS-5932
Learn how to maximize the power of your code in your M3G applications.
It may not work on some devices, due software/hardware implementation.
2008 JavaOneSM Conference | java.sun.com/javaone | 2
Agenda
Basic M3G Meshes Texturing Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone |
3
M3G is not Java 3D™ API
Scene Handling Compile Method Locale TransformGroup Sound M3G Simple Camera Java 3D
ViewPlatform, PhysicalBody…
Much Simpler Less Processing Fast Rendering
2008 JavaOneSM Conference | java.sun.com/javaone | 4
M3G first tips:
Try to cheat as much as you can • Come up with different ways to solve the same problems • Your goal: Performance and resource consumption….but it comes with
a price.
The number of polygons is not the ONLY measure way • The whole area to be painted should be taken into consideration • A small model with many polygons can actually draw faster than a big
model with fewer polygons.
If possible use quads instead of triangles when you create a 3D models Use small number of textures of possible • Make sure to use a single 256*256 texture in preference to using four
with a size of 128*128
2008 JavaOneSM Conference | java.sun.com/javaone | 5
M3G first tips:
Have a good 3D designer • Tools: 3D Max, Blender Remember: • Memory size and computation resource are limited
View
M3G app
Effects
Performance
2008 JavaOneSM Conference | java.sun.com/javaone | 6
M3G: 3 Steps Process
It can be done with 2 people:
Developer and 3D Designer
Modeling
Scene Management
Animation
2008 JavaOneSM Conference | java.sun.com/javaone |
7
M3G Lifecycle
YourMIDlet
public class YourMIDlet extends MIDlet { private YourCanvas3D yourCanvas; public void start() { …
public class YourCanvas3D extends GameCanvas { private private private private Graphics3D iG3D; Camera iCamera; Light iLight; float iAngle = 0.0f;
YourCanvas3D Setup Stage Rendering Stage
private void init() throws Exception { …
protected void paint(Graphics g) { Graphics3D g3D = Graphics3D.getInstance(); g3D.bindTarget(g); g3D.setCamera(iCamera); g3D.addLights(lights); g3D.render(myVertex, iIb, iAppareance, iTransform); g3D.releaseTarget(); }
2008 JavaOneSM Conference | java.sun.com/javaone |
8
M3G Lifecycle
YourMIDlet
Peak Memory Consumption occurs here
Tip:
Only initialize and load the objects that you sure are needed.
Example: Load an PNG Image for Texturing
YourCanvas3D PNG Setup Stage Rendering Stage JAR
decompress
copied
Image
Image2D
Worst case scenario: 1 compressed and 2 uncompressed
Trick: If running out of memory during loading,
try split into parts and load separately.
2008 JavaOneSM Conference | java.sun.com/javaone | 9
M3G Lifecycle
YourMIDlet
Rendering is a Singleton
Tip:
Do not set another thread, expecting things run faster
YourCanvas3D Setup Stage Rendering Stage
g3D.bindTarget(g); g3D.render(); g3D.releaseTarget();
Once things are bound….
Tip:
Do not make changes to a bound Object, it can generate unpredictable results Conference | java.sun.com/javaone | 2008 JavaOne
SM
10
Agenda
Basic M3G Meshes Texturing and Lights Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 11
Meshes tips:
Try mix 2D/3D: • In certain situations, you can mix 2D content • 2D consumes less memory • Drawing directly in the background image will also reduce resource
consumption
When modeling, always retain the unit scale to choose for the model • All exporters seem to have different ways of handling a unit Some devices, has a limit of amount polygons of a 3D scene • Example: Motorola E680/E680i/A780 Divide large mesh into several part, but a balance should be made (only by experimentation): • More parts, more processing is required • Less parts, Large objects that are only partly visible can not be culled.
2008 JavaOneSM Conference | java.sun.com/javaone | 12
VertexArray
(-10,0,0) (0,0,10)
Y
(0,10,0) (0,0,-10)
(10,0,0)
X
Z
// Every set of three elements of this // array, gives a vertex of the pyramid.
short[] vertices = {0, 0, 10, 10, 0, 0, 0, 10, 0, 0, -10, 0, -10, 0, 0, 0, 0, 10, 0, -10, 0, 10, 0, 0, 0, 0, 10};
2008 JavaOneSM Conference | java.sun.com/javaone | 13
VertexArray
// Here we construct the VertexArray, which is a generic // data structure for storing collections int numVertices = vertices.length/3; VertexArray vertexArray = new VertexArray(numVertices,3,2); // set the data, starting from index 0 vertexArray.set(0, numVertices, vertices); // Create a 3D Object
VertexBuffer vertexBuffer = new VertexBuffer(); vertexBuffer.setPositions(vertexArray, 1.0f, null);
2008 JavaOneSM Conference | java.sun.com/javaone | 14
VertexArray
// Here we construct the VertexArray, which is a generic // data structure for storing collections Tip: int numVertices = vertices.length/3; Create your vertex during setup stage and only VertexArray vertexArray =modify then when you really need to. new VertexArray(numVertices,3,2); // set the data, starting from index 0 vertexArray.set(0, numVertices, vertices); // Create a 3D Object
You can use set at any time to modify the contents or change the arrays being used….but it may not be cheap.
VertexBuffer vertexBuffer = new VertexBuffer(); vertexBuffer.setPositions(vertexArray, 1.0f, null);
2008 JavaOneSM Conference | java.sun.com/javaone | 15
VertexArray
VertexBuffer has a bias parameter for normal texturing // Here we construct the VertexArray, which is a generic
// data structure for storing collections Tip: int numVertices = vertices.length/3; Always VertexArray vertexArray = leverage this parameter, because avoid unnecessary performance penalty, new VertexArray(numVertices,3,2); especially on software-only implementations. // set the data, starting from index 0 vertexArray.set(0, numVertices, vertices); // Create a 3D Object
VertexBuffer vertexBuffer = new VertexBuffer(); vertexBuffer.setPositions(vertexArray, 1.0f, null);
2008 JavaOneSM Conference | java.sun.com/javaone | 16
IndexBuffer and Triangle Strips
// Here we define the triangle strips // use the first six vertices to make one strip of // triangles, then make a triangle strip from the next // three vertices int[] strip = {6,3}; // Then construct the corresponding IndexBuffer // as an implicitly-defined TriangleStripArrat IndexBuffer indexBuffer = new TriangleStripArray(0, strip);
2008 JavaOneSM Conference | java.sun.com/javaone | 17
IndexBuffer and Triangle Strips Triangle strips are very efficient, but there is a
considerable associated when rendering each strip.
// Here we define the triangle strips Tip: // use the first six vertices to make one strip of Join several strips together using degenerate // triangles, then make a triangle strip from the next triangles, duplicating the end index of the // three vertices first and the beginning index of second\ int[] strip = {6,3}; // Then construct the corresponding IndexBuffer // as an implicitly-defined TriangleStripArrat IndexBuffer indexBuffer = new TriangleStripArray(0, strip);
2008 JavaOneSM Conference | java.sun.com/javaone | 18
Mipmapping
Trick: Always enable mipmapping. Not only does it make
your graphics look better, it also saves valuable memory bandwidth.
Choosing between FILTER_NEAREST and FILTER_LINEAR is a valid trade-off between quality and performance.
2008 JavaOneSM Conference | java.sun.com/javaone | 19
Alpha Testing
The fastest way to discard individual fragments. Your rendering speed may improve by using alpha testing • It discards transparency before the blending stage. Many Implementations (in particular software ones) detect earlier in the rendering pipeline.
2008 JavaOneSM Conference | java.sun.com/javaone | 20
Disable color writes
It can force certain hardware implementations into hugely expensive workarounds.
Trick: First make sure that all objects in the scene
have alpha writes enabled.
2008 JavaOneSM Conference | java.sun.com/javaone | 21
Agenda
Basic M3G Meshes Texturing and Lights Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 22
Texture and Lighting tips:
Use light maps for lighting effects Add a pre-calculated lighting to a scene in 3D max Multitexturing is your friend Use perspective correction hint • Faster than adding more vertices Use background images • Scale, tiled and scroll Use sprites
2008 JavaOneSM Conference | java.sun.com/javaone | 23
Camera
// Create the camera object to define where the polygon // is being viewed from and in what way: Camera camera = new Camera(); // Set the camera so that it will project the 3D // picture onto the screen in perspective, with a // vanishing point in the distance. camera.setPerspective(60.0f, (float)getWidth()/(float)getHeight(), 1.0f, 1000.0f);
2008 JavaOneSM Conference | java.sun.com/javaone | 24
Light
// Coordinates of normal vectors, each one corresponding // to a vertex short[] normals = {0,0,10, 10,0,0, 0,10,0, 0,-10,0, -10,0,0, 0,0,10, 1,-1,1, 1,-1,1, 1,-1,1}; // Place the values for the normals in a VertexArray VertexArray vertexNormals = new VertexArray(numVertices, 3, 2); // Set the data, starting from index 0 vertexNormals.set(0, numVertices, normals); // Set the normals for the VertexBuffer vertexBuffer.setNormals(vertexNormals);
2008 JavaOneSM Conference | java.sun.com/javaone | 25
Light
// Now add the light Light light = new Light(); light.setMode(Light.AMBIENT); light.setIntensity(20.0f); // Show some reflection Appearance appearance = new Appearance(); // First a reflective material Material material = new Material(); material.setShininess(100.0f); appearance.setMaterial(material);
2008 JavaOneSM Conference | java.sun.com/javaone | 26
Light
// Now add the light Light light = new Light(); light.setMode(Light.AMBIENT); light.setIntensity(20.0f); // Show some reflection Appearance appearance = new Appearance();
In general, Lighting is rather complex. // First a reflective Spot lights and attenuation are particularly material Expensive.
Material material = new Material(); material.setShininess(100.0f); appearance.setMaterial(material); Tip:
Think hard before decide for lighting
2008 JavaOneSM Conference | java.sun.com/javaone | 27
Light
// Now add the light Light light = new Light(); light.setMode(Light.AMBIENT); light.setIntensity(20.0f);
Different light types have different runtime performance costs. // Show some reflection AMBIENT: Appearance appearance = new Appearance(); free DIRECTIONAL: cheap OMNI: expensive // First a reflective material SPOT: expensive
Material material = new Material(); material.setShininess(100.0f); Tip: appearance.setMaterial(material); Use OMNI and SPOT only when
absolutely necessary.
2008 JavaOneSM Conference | java.sun.com/javaone | 28
Texture
// Define the coordinate values short[] textures = {0,0, 2,2, 2,0, 4,2, 4,0, 2,2, 1,1, 1,0, 0,1}; // Place them in a VertexArray, indicating that // each vertex gets two coordinates stored on two // bytes each: VertexArray vertexTexture = new VertexArray(numVertices, 2, 2); vertexTexture.set(0, numVertices, textures); // Set the coordinates in the VertexBuffer with // the other coordinates, without scaling vertexBuffer.setTexCoords(0, vertexTexture, 1.0f, null);
2008 JavaOneSM Conference | java.sun.com/javaone | 29
Images as Textures
// Start by creating the 2D images Image image = Image.createImage(“/images/hello.png”); Image2D helloDiagonal = new Image2D(Image2D.RGB, image); image = Image.createImage(“/images/hello_trans.png”); Image2D helloTransparent = new Image2D(Image2D.RGBA, image); // Create the corresponding texture objects Texture2D texture = new Texture2D(helloTransparent); // set the image to repeat texture.setWrapping(Texture2D.WRAP_REPEAT, Texture2D.WRAP_REPEAT); texture.setBlending(Texture2D.FUNC_MODULATE); // set the image to repeat appearance.setTexture(0, texture); 2008 JavaOne Conference | java.sun.com/javaone | 30
SM
Images as Textures
// Start by creating the 2D images Image image = Image.createImage(“/images/hello.png”); Like background images, most implementations simply Image2D helloDiagonal = new Image2D(Image2D.RGB, image); Implement Sprite3D using textured quads. It limits the texture of images apply. image = Image.createImage(“/images/hello_trans.png”); Image2D helloTransparent = new Image2D(Image2D.RGBA, image); Tip:
To maximize performance an quality, stick texture image restrictions // Create the corresponding to the textureobjects with sprites as well. Texture2D texture = new Texture2D(helloTransparent);
// set the image to repeat texture.setWrapping(Texture2D.WRAP_REPEAT, Texture2D.WRAP_REPEAT); texture.setBlending(Texture2D.FUNC_MODULATE); // set the image to repeat appearance.setTexture(0, texture); 2008 JavaOne Conference | java.sun.com/javaone | 31
SM
Agenda
Basic M3G Meshes Texturing and Lights Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 32
Scene Tips:
Make sure you only include the data you really need in your Mesh objects • Some tools adds vertex normals by default Your camera must be part of your World • If not, it may thrown an exception If you want your World to be garbage collected: • It will not happen as long as it camera and lights are referenced • g3D.setCamera(null, null) and g3D.resetLights() Some early M3G implementations lack sufficient numeric range and precision to compute alignments accurately.
2008 JavaOneSM Conference | java.sun.com/javaone | 33
World
World myScene = new World(); myScene.setBackground(myBackground); Group myAuto = new Group(); myAuto.addChild(myBike); for(int i=0; i < 5; i++) { myAuto.addChild(myCarWheel[i]); myCarWheel[i].setTranslation( wheelX[i], wheelY[i], wheelZ[i]); }
2008 JavaOneSM Conference | java.sun.com/javaone | 34
Setting Up Camera and Light
myAuto.addChild(myCar); myAuto.setScale(0.1f, 0.1f, 0.1f); myAuto.setOrientation(-30.0f, 0.0f, 1.0f, 0.0f); Camera camera = new Camera(); myScene.addChild(camera); myScene.setActiveCamera(camera);
2008 JavaOneSM Conference | java.sun.com/javaone | 35
Agenda
Basic M3G Meshes Texturing and Lights Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 36
Animation Tips
M3G file offers a choice between 16-bit and 32-bit • Check at your target platform supports You can speed up animation process by telling M3G to ignore any animations you do not need at any given moment. • Set the weight of the respective controller to ZERO • Or set the controller of an animation track to NULL Sometimes, you can use textures to make them display realistically.
2008 JavaOneSM Conference | java.sun.com/javaone | 37
Basic Animation
KeyframeSequence
AnimationTrack Object3D
Positioning
AnimationController
Base of all classes that can be animated
Active time, speed, etc..
AnimationTrack
Orientation
KeyframeSequence
2008 JavaOneSM Conference | java.sun.com/javaone | 38
KeyframeSequence
// Define a object to hold a series of six frames KeyframeSequence seq = new KeyframeSequence(6,3, KeyframeSequence.LINEAR); // Define a series of values for the key frames
seq.setKeyframe(0,0, new float[] {0.0f, 0.0f, -1.0f}); seq.setKeyframe(1,1000, new float[] {3.0f, 0.0f, -2.0f}); seq.setKeyframe(2,2000, new float[] {6.0f, 0.0f, -3.0f}); seq.setKeyframe(3,3000, new float[] {4.0f, 0.0f, -5.0f}); seq.setDuration(10000); You can always read the values back.
Tip:
Use the animation for anything
2008 JavaOneSM Conference | java.sun.com/javaone | 39
AnimationTrack
// Wrap the keyframe sequence in a animation // track that defines it to modify the translation AnimationTrack at = new AnimationTrack(seq, AnimationTrack.TRANSLATION); // Have this track move the group myGroup.addAnimationTrack(at); // Initialize an animation controller to run the // animation AnimationController ac = new AnimationController(); at.setController(ac); ac.setPosition(0, 0);
2008 JavaOneSM Conference | java.sun.com/javaone | 40
Agenda
Basic M3G Meshes Texturing and Lights Scene Graph Animation Conclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 41
Conclusions
Next Steps
Remember: • Your device has resources constraints • To check on Manufacture’s developer site for more tips&tricks Some devices are coming with Graphics Acceleration Capabilities • Sony-Ericsson W800i • Nokia N-Gage • …Many more devices to come M3G are mainly used for Games • But it can also use to enhance user experience • Data visualization
2008 JavaOneSM Conference | java.sun.com/javaone | 42
Summary
JSR 184 (aka M3G) is an API for 3D graphics on mobile devices Huge differences from Java 3D Easy to develop low-level meshes (immediate mode) Easy to develop high-level meshes (retained mode) Improves the user interface
2008 JavaOneSM Conference | java.sun.com/javaone | 43
For More Information
See • Mobile 3D Graphics with OpenGL ES and M3G, by
• • • • • •
K.Pulli,T.Aarnio,V.Miettinen,K.Roimela and J.Vaarla Creating Mobile Games, by Carol Hamer Sony-Ericsson Development Site • http://developer.sonyericsson.com/site/global/home/java_3d/p_java3d.jsp Motorola Development Site (3D Performance) • http://developer.motorola.com/docstools/technicalarticles/ www.conversay.com www.conversations.com for the Developer Network JSR references • http://www.jcp.org/184
2008 JavaOneSM Conference | java.sun.com/javaone | 44
Mauricio “Maltron” Leal
Developer Community Maven
Sun Microsystems Learning Services – Technology Group
TS-6258
2008 JavaOneSM Conference | java.sun.com/javaone | 45