Frequently Asked Questions¶

How do I set up a Python 3 distribution to work at home?¶

Practicals physically take place on the lab machines at Ensimag and UFRIm2ag. You don’t need any of these steps on lab machines as they are already pre-configured.

The instructions here are provided for convenience, and support will be provided only as best-effort outside of regular TP hours, or for cases of first necessity.

These instructions are a work in progress. If you have further details for a specific platform, we can put them up here.

If you encounter a problem, first read the troubleshooting section below. If that fails, when seeking assistance from your instructors, please be specific by indicating the platform, GPU and CPU chipset, system (including distribution, version), the libraries and Python versions (particularly for glfw and assimp), commands typed, and shell output with the error.

(Windows 10/11 only): install WSL 2, select Debian or Ubuntu, then
```
sudo apt-get install mesa-utils
```
Download and install the VcXsrv X server Windows application, execute it, then uncheck “Native OpenGL” and check “Disable Access Control” in “Extra settings”.

Add the following to your .bashrc (or .zshrc) to tell your linux graphical applications to use the X server above:
```
export DISPLAY=$(awk '/nameserver / {print $2; exit}' /etc/resolv.conf 2>/dev/null):0
```
Then proceed below under Debian/Ubuntu.

Install Python >= 3.5 with a pip installer, GLFW and Assimp, and clang, using your package manager.

Under Linux Debian, Ubuntu or derivatives:

sudo apt-get install python3 python3-pip libglfw3-dev libassimp-dev

Under Linux Arch, Manjaro or derivatives:

sudo pacman -S python python-pip assimp glfw-x11

Under CentOS or RedHat derivatives:

sudo yum install python3 python3-pip assimp glfw

Under MacOSX using Homebrew:
```
brew install python3 glfw assimp llvm
```

Under MacOSX using MacPorts:

sudo port install python38 py38-pip glfw assimp clang-11

On many recent distributions, python3 is the default python distribution, e.g. Arch/Manjaro, and Ubuntu >= 20.4. You can check this with the command python --version. In this case simply replace python3 and pip3 with python and pip in the following instructions.

With the pip3 installer, type:
```
sudo pip3 install numpy Pillow PyOpenGL PyOpenGL-accelerate glfw cython AssimpCy
```
Run your viewer.py script:
```
python3 viewer.py
```

Troubleshooting¶

If Python halts saying: unknown symbol or binary operator ‘@’, you’re running a Python version < 3.5
As explained in the wrappers section below, PyOpenGL, glfw and AssimpCy are Python wrappers that rely on the OpenGL, GLFW and assimp shared binary libraries. The most frequent error is if the wrappers don’t find their corresponding shared binary library. Here are a few instances:
- on some unix platforms (including MacOS), if you see an exception:
```
ImportError: Failed to load GLFW3 shared library
```
  Then your GLFW shared library is probably in a non-usual location that the python wrapper hasn’t looked up. You need to locate where it is, then use the following command to indicate the full path of the library file, e.g. if it is in /opt/local/lib:
```
export PYGLFW_LIBRARY=/opt/local/lib/libglfw.dylib
```
- on MacOS Big Sur / Monterey, the following exception may occur:
```
ImportError: Unable to load OpenGL library
```
  OpenGL has been moved to a different path and the wrapper hasn’t yet caught up to look for it in the new location. There is a temporary fix here.

Upon compiling AssimpCy, if you see the following compilation error:
```
Cannot open include file: 'types.h'
```
Then the AssimpCy setup.py script wasn’t successful in locating where you installed your assimp header and library files.

Are you sure you performed step 1 above to install the assimp binaries?

If it still doesn’t work, you can attempt a manual install of AssimpCy:
```
git clone https://github.com/jsfrancal/AssimpCy.git
cd AssimpCy
python3 setup.py build_ext
sudo python3 setup.py install
rm -rf AssimpCy
```
If you still have the error with ‘types.h’, you can inquire your package manager where it installed the assimp library and include files (e.g. dpkg -L libassimp-dev on Debian), then explicitly inform setup.py of their location as follows:
```
python3 setup.py build_ext -I'path/to/assimp/headers' -L'path/to/library/'
```
Upon compiling AssimpCy, if you see a compilation error:
```
file not found ./assimpcy/all.cpp
```
Then you forgot to install Cython:
```
sudo pip3 install cython
```

What does the load function do, in core.py?¶

The load() function in the provided core.py is a function whose purpose is to load a 3D model file’s objects and all their possible attributes and textures.

It uses the AssimpCy library to load many supported file formats such as OBJ, FBX, GLTF, GLB, Collada (*.dae)…

Many of these formats can store a complete scene and even skeletal animations (see the optional Bonus - Skeletal Animation to see how to load those). This implies that the file contains a full hierarchy of mesh objects. The call to the load() function returns this as a full tree of Node and Mesh objects, where Node instances materialize hierarchical transforms and may contain an arbitrary list of Mesh or more Node objects, as seen in Practical 3 - Hierarchical modeling.

The load() function also loads texture coordinates as an attribute and instantiates Texture objects and wraps the appropriate mesh objects with Textured decorators, as seen in Practical 5 - Texturing.

More generally any attribute present for a given mesh are created and passed to the Mesh constructor, but they follow a strict naming convention which you need to follow exactly when receiving the attributes in your shaders, as follows:

in vec3 position is a vector attribute with the current vertex coordinates, always
in vec3 normal is the normal vector attribute per vertex, if present in the file
in vec2 tex_coord is an attribute for the 2d texture coordinates of the vertex, if present in the file
in vec3 color is an RGB attribute with an optional per-vertex RGB color, if present in the file
in vec4 bone_ids and in vec4 bone_weights are skeletal attributes that are explained in Bonus - Skeletal Animation, present if the file contains an animation.

The load() function also defines a number of uniform variables reflecting what is the the 3d model file, again with a strict naming convention to be followed in the shader:

k_d, k_s, k_a, and s, are exactly the diffuse, specular, ambient (uniform vec3 k_*) and shininess (uniform float s) constants of the Phong illumination model, seen in Practical 4 - Local Illumination.
if a texture is associated to the given Mesh object, the texture automatically created will be bound to a uniform sampler2D diffuse_map.

The binding for all the variables comes for free for objects created with the load() call, you can directly receive them on the shader side with no specific code on the CPU side, and simply the right variable declaration in your shader on the GPU side. If an optional variable was not found in the file, the corresponding attribute or uniforms will simply return 0 values if declared in your shader. Conversely your shader doesn’t need to load every variable present in the file. If the variable isn’t there in your shader, it simply won’t be bound.

Recall that by convention, we also impose the names of the model, view, and projection matrices, which are passed as uniform mat4 by the Viewer object at each frame.

Last but not the least, note that the load() function requires at least two parameters:

file: string containing a path to the 3d model/scene file to be loaded.
shader: Shader instance to be used for rendering.

And it also accepts optional parameters:

tex_file allows the user to load a texture, to override whatever texture was specified in the 3D model file
all remaining keyword arguments passed to load() will be bound in the shader as uniforms, which allows to pass custom variables to the constructor of all mesh objects created by the load() call, which will be seen in the shader when rendering them.

Models don’t load, what is going on?¶

Many models found on the internet have bogus formats and non standard paths. Assimp also fails to load some format variants. The assimp loader code provided with the practicals is also simplified under certain assumptions of the file.

To add functionality and/or debug what is going on with assimp, you can use the following script which dumps the content of the AssimpCy structure loaded for a given 3D file on the command line.

Download assimp-dump.py

How do I make my code faster, I want higher FPS?¶

First, let us clear one common misconception: Python is not too slow to run a graphics loop, but our code base is optimized for ease of use, not speed. To have an idea of where your program is spending time, take a look at Time your code for performance.

A code optimized for speed in Python needs to target AZDO principles (Approaching Zero Driver Overhead, Advanced Talk). With these principles, ultimately it is possible to draw all objects in a dynamic scene with just a few OpenGL bind calls and down to only one OpenGL draw call per frame.

Not all of these changes are realistic to implement within this project as they would require a complete re-design of the architecture, but we list the most accessible of them by order of difficulty.

Generally: the idea is to minimize the number of state changes, OpenGL calls and objects in the scene hierarchy in the inner draw loop.

(very easy) never load the same object twice from disk or create two separate Mesh objects with the exact same Mesh data. Use the same Python object instance and place it into different Nodes in the hierarchy if needed.
(easy) for static, non-moving objects that are part of your scene and can share material parameters and texture (e.g. all instances of static trees that reference a single texture), pre-transform their vertices once, merge them into a single, pre-baked object (one Python Mesh object => one OpenGL draw call for the group)
(moderate) Replace Python matrices and vectors with PyGLM vec and mat objects which are highly optimized.

Advanced, bindless real-time rendering

The basic idea is to take what initially was a hierarchy of different Mesh objects and group them as sub-objects inside a single Mesh object instance, that will act as a container. The OpenGL / draw calls for any of the refactored features below will then be emitted once only for the container Mesh instance, and not per sub-object.

(very easy) Set scene globals (lights, camera parameters) only once at the mesh container level (as a parameter of container Mesh constructor if static, or as parameter of the container Mesh draw method if updated per frame).
(easy) Put all your different material parameters for all objects into a single uniform array initialized only once, setting it as parameter of the containing Mesh constructor call. For each sub-object vertices, pass an extra object index per-vertex attribute so that the shader calls for that object can go fetch the right material in the array GPU-side without having to explicitly set a GLSL material variable from the CPU with per-sub-object OpenGL calls.
(moderate) Put all your dynamic transforms (which need to be recomputed per frame) for all objects in a single uniform matrix array, and set this array once per frame as parameter of the draw method of the container Mesh class above. Again use the previously mentioned per-sub-object-vertex index attribute so that the shader calls for that object can get the right transform matrix from the array GPU-side, with no per-sub-object OpenGL calls. Note: you may need to manage sub-object transform matrix updates differently than through the current Node hierarchy).
(hard, requires creating an ArrayTexture class from scratch) Put all of your textures into a single Array Texture object at container Mesh initialization (need to resize them all to a single identical size and type first). Use aforementioned object index attributes or pass an extra texture index as per-sub-object vertex attribute so that the shader calls for that object can go fetch the right texture in the array texture without having to explicitly set it on the CPU side with per-sub-object OpenGL calls.
(quite hard, as it requires the previous 4 points) Use only one global mesh container with all your scene objects as sub-objects, and a single so-called “uber-shader” that accounts for all rendering cases. Stack all vertex attributes from your different sub-objects into only one set of attributes passed to the constructor of the container Mesh. Although it has sub-objects with different transforms, materials and texture, you can now render your whole scene with one draw call of the container Mesh. All draw calls of sub-objects can be removed.
or (variant of the above, still quite hard) group scene objects that have common rendering characteristics and a common shader into one corresponding Mesh container. The scene contains only as many Mesh container instances as there are different needed shaders and attribute / parameter profiles. You still need to stack sub-object attributes into one big attribute array for each container as described above. The scene can now be rendered with a constant number of Mesh container draw calls, independent of the total number of sub-objects.
(quite hard, requires adding a use case to VertexArray and probably creating a specific InstancedMesh class) if you want to render many dynamic instances of the same geometry, such as moving particles, look up OpenGL instanced rendering. This would be a new, specific type of drawable Mesh-like InstancedMesh object.

How do PyOpenGL / pyGLFW wrappers work?¶

This is not mandatory to start the practicals but helps understand their Python mechanics.

Python wrappers simply load the original C-based dynamic libraries (libGL.so, libglfw.so under Linux) and call the binary function code within when you call the corresponding Python function. Typically wrappers add a layer of Python code which hides all this to the user, and makes some conversions from Python objects to C-structures and vice-versa, for Pythonic convenience.

For example, OpenGL being a C API, typical C OpenGL functions take arguments either as built-in C-types (such as int, float, char…) or pointers to C arrays of these types which have a flat, contiguous layout in memory.

This is why PyOpenGL and Numpy are such a great match: Numpy objects are designed to represent multi-dimensional arrays of basic types, stored internally as a contiguous memory chunk with flat memory layout. So when you pass Numpy arrays to PyOpenGL as intended, all it has to do is get the pointer to that internal memory chunk and pass it to the corresponding C function as argument. Done.

Consider for example the C specification of the following OpenGL function, designed to pass count vector of 4 float values to a shader variable in OpenGL:

void glUniform4fv(GLint location, GLsizei count, const GLfloat *value);

(GLint, GLsizei and GLFloat are OpenGL typedefs for int, unsigned int and float). You would typically call it from a C code as follows to pass one vector of 4 floats:

#include <GL/gl.h>
... // retrieve location of shader variable to update
GLfloat my_value[4] = {1., 2., 3., 4.};
glUniform4fv(location, 1, my_value);

So a Python equivalent of the C-function call above is:

import numpy as np
import OpenGL.GL as GL
...  # retrieve location of shader variable to update
my_value = np.array([1, 2, 3, 4], np.float32)
GL.glUniform4fv(location, 1, my_value)

But PyOpenGL goes further than that to make things more Python friendly. For most C functions taking an array of values, you can pass Python iterables to the corresponding Python function, such as a tuple or list. PyOpenGL detects that and automatically converts it to a flat layout buffer to pass it to the underlying C function:

GL.glUniform4fv(location, 1, (1, 2, 3, 4))

With these rules in mind, you can now basically read the C documentation of the OpenGL and GLFW APIs and directly infer how to call them in Python.

Note

If the conversion were not possible for some reason, for example if the tuple given in the call was made of heterogeneous or non-Number types, PyOpenGL would raise an exception here.

Error management. Python wrappers offer another service for considerable time gain and ease of use. In the original C APIs for OpenGL and GLFW, the user must explicitly and regularly call some functions to detect and retrieve an error state about OpenGL or GLFW calls, because there is no exception mechanism in C. Both PyOpenGL and pyGLFW internally wrap every OpenGL call with this error checking, and converts error occurrences to Python exceptions, such that no systematic and explicit error checking code is necessary.