It was a busy and exciting week for MAD amid the launch of the new website. On Monday, we still managed to meet up with Nikolai and Nicholas to talk about the possible visual directions for the data visualization.

I was encouraged to keep the visual outcome a bit open at this point. Instead of choosing to focus on a couple of basic geometries or a painting-like shader, Nikolai suggested that I explore as much as the timetable allows. We agreed that the NLP side of the text visualization app needs to be worked on simultaneously, but for the next two weeks it makes sense to focus on understanding Three.js and shaders as they are not as familiar to me. After creating a few visually appealing animations, it is then easy to choose the most interesting one and connect that to data.

I created a bunch of HTML files that I could fill with Three.js sketches and then just open them as tabs in the browser. One of the benefits of Three.js is that it can simply be included as a script in the HTML:

<script src="js/three.js"></script>

…And then you are ready to create 3D scenes!

Simulating data

I decided to simulate data by creating a set of random values that could be coming from the NLP and server side:

var value1 = Math.floor(Math.random() * 2001); // Could be number of tweets
var value2 = Math.floor(Math.random()); // between 0 and 1
var value3 = getRandomArbitrary(-1.0, 1.0); // Could be sentiment value between -1.0 and 1.0
var value4 = Math.floor(Math.random() * 21); // Could be average sentence length
var value5 = Array.from({ length: 2000 }, () => Math.floor(Math.random() * 4)); // Could be certain values of all the accounts 2000 tweets

I experimented with Three.js examples on groups, buffer geometries and lines. For example, I managed to make cube colors depend on an array called randomValues, its length being the same as the number of the cubes.

let randomValues = Array.from({ length: numberOfCubes }, () => Math.floor(Math.random() * 10));

Each index can have a value between 0-9; if the value is more than 5, color is violet. This could be a simple way to illustrate the amount of negative or positive tweets in the user account. In a similar fashion, I experimented with this Three.js example by changing the colors and the number of triangles based on my variables:

First steps with shaders

In addition to this, I had an exciting shader workshop with Nikolai. I have previously used some vertex and fragment shaders in openFrameworks. There, shaders live as separate files and are loaded when the app compiles. However, we first focused on understanding how to make GLSL, ie. the language of shaders, communicate with Javascript in a web app.

As an example, we inspected the shader code Nikolai had written for the background on the MAD website. The goal for me was to understand how to get fictional variables data1, data2 and data3 to have an effect on both the shader AND the Three.js side of a visual. For example, the color and pattern of the shader could be affected by the sentiment values of the Twitter account and the size of the 3D shapes by something else.

In practice, shader outputs can be controlled by using uniforms. They are global shader variables that can be updated in the animate() function on Javascript side, for example.

A gradient shader with some Tuple drawings about noise

We had fun tweaking colors and the amount of noise and graininess. With shaders, it is easy to get rewarding results by accident. However, I wanted to understand them better in order to actually control the visuals.

We also discussed the balance between making a data-based, abstract artwork and a visual that strictly represents data (each shape is of accurate size etc.). I’m after a result that is somewhat in between. We discussed that one can illustrate the character of a text via colors, patterns, animation speeds and so on – the result is an interpretation.

I will study these two shader resources before the next weeks' session: