Intro

Using webgl shaders is a great way to bring your webpage to life with highly interactive and creative visual effects. Libraries like Martin Laxenaire’s Curtains.js  make it possible to Render a whole HTML website in WebGL

‍

But while rendering a whole website in webgl looks awesome on performant devices, it can quickly become an absolute nightmare to optimise across devices and browsers. Low framerates, illegible pixelated text and scroll jank are the main culprits that will turn a cutting edge website into an unusable mess.

‍

What makes these issues especially tricky to address, is that fixing one will often exacerbate the others.

‍

In order to get some context for these issue and their solution, let’s take a look at a project where we had to tackle all three of these issues:

‍

The Case Study

‍

Beyond Studio is a Virtual Worlds Venture Studio, creating immersive experiences across a variety of immersive digital mediums and emerging business models.

‍

Having undergone a recent brand refresh, they approached Psychoactive to create a website that would utilise their new visual identity to showcase their flagship projects, as well as the studios’ talented creative team, their story, values, way of working, and news.

‍

The site had to reflect the studio’s innovative and creative nature, through a highly interactive and visually engaging web experience, while offering an intuitive and user friendly experience.

‍

The “lens” shape from their logo had to feature prominently in the experience, acting as a filter to reveal and morph imagery from the various worlds they create.

‍

Brief Overview of the Tech

The website itself was built using the Webflow platform, offering the beyond team a built in cms to edit all the imagery and text content.

‍

The WebGl functionalities were developed using the coding environment outlined in our other article : Better Webflow Custom Code With VS Code, Node.Js, Parcel.Js & GitHub

‍

Curtain.js was used to ensure a perfect alignment between WebGl elements and their corresponding DOM html elements.

‍

This lightweight yet powerful library makes it relatively easy to automate this “DOM to WebGL” process, rendering an entire HTML website in WebGL, including text, as outlined by the library’s creator Martin Laxenaire in his great medium article : Portfolio 2021 technical case study — Rendering a whole HTML website in WebGL

‍

‍

he “lens” filter effect is achieved using two WebGLRenderer contexts :

1. A curtain.js one similar to this codepen to handle all the images and text content that needs to be affected, as well as all the animated gradients
2. A three.js scene, to handle the motion and interaction with the puck

‍

The three.js canvas is then loaded as a uniform sampler2D into our curtain.js custom WebGL fragmentShader using the shaderPass.loadCanvas() function.

‍

The site also makes use of the anime.js  timeline to animate the position and behavior of the “lens” and the colors of the background as the user scroll downs the page.

‍

Optimisation with Adaptive Resolution

‍

While the 3D scene is pretty lightweight, rendering it on top of a pretty complex WebGL shader tends to require some optimisation, especially on devices with high pixel density. In order to keep the app responsive and usable, it is crucial to prioritise a high framerate over visual quality.

‍

The most effective solution for maintaining 30+ fps is to downres the whole WebGL resolution. However detecting how well a user’s device can run the experience is very tricky. While libraries like detect-gpu  can give a benchmark estimate, they don’t always give accurate readings depending on the web browser used - and there is no way to know how many background apps the user will be running while on our site.

‍

The most foolproof way to know how well our site is running, is to implement frame-rate monitoring, and adjust the quality/resolution based on how many fps the user is getting.

‍

For example, we might use detect-GPU to estimate the site can render at 1080p, then check every couple seconds if our fps is below 30, if it is, we can incrementally down-res the webgl down to 720p or until we get a steady 30fps.

‍

While slightly more pixelated images are usually fine, especially when they are also being morphed by our WebGL shader, pixelated text quickly becomes illegible.

‍

The solution here seems pretty simple : don’t render text in WebGL if the resolution is too low, just  leave it as DOM elements over the webGL canvas - browsers have no problem rendering crisp text content for DOM elements. But as soon as we have DOM elements scrolling next to WebGL content, the whole experience will appear incredibly janky.

‍

The Scroll Jank

So what is this jank, and where is it coming from ?

‍

Fundamentally the WebGL render context is completely separate from the way our browser renders and scrolls the DOM, and even with a smooth 60FPS, any mismatch in the speed of the content scrolling will make the site feel janky. This will be greatly exacerbated at lower frame rates, where the site might becoming completely unusable as DOM elements could zoom past or completely overlap WebGL elements they are meant to line up with.

‍

The unpredictable frame-rate limitation of our WebGL context means we can’t perfectly match it to our default DOM scrolling. The only way to fix this issue is to scroll the DOM in perfect sync with the WebGL - overriding the browser’s default scroll behavior.

‍

To understand how to do this, let’s first look at how we usually tackle on-scroll animations in javascript:

‍

Usually we keep the browser’s default scroll behaviour, and listen to window/DOM scroll event using something like window.addEventListener(”scroll”, onScrollFunction) to run a function that will update the scroll position of our WebGL/curtain.js scene.

‍

While it’s gotten significantly better in recent years, when and how this “scroll” event listener triggers can be very inconsistent across browsers and input devices. A lot of issues can arise, so it’s usually recommended to throttle and/or debounce our scroll function to ensure it’s not being triggered too quickly and to make it more predictable.

‍

But it won’t cut it in our case, because we still don’t have control over how and when the browser’s will scroll the DOM.

‍

The Solution : Virtual Scrolling

‍

he way to fix this actually fairly straightforward:

• Make the DOM element unscrollable, by putting our scrollable dom elements in a full-screen container div with overflow:hidden
• Use the virtual-scroll library to capture when and how far the user scrolls (this library listen to touch / scroll wheel / keyboard input, not the DOM’s scroll event)

this.scroller = new VirtualScroll( {preventTouch: false, touchMultiplier : 3, mouseMultiplier : 0.5})

this.scroller.on( event => {
	this.onScroll(event)
})

• Calculate how far through the container we have scrolled, making sure to clamp our values to 0 and thee height of our container div

onScroll(e){
	if(this.canScroll){
	//we can turn off scrolling when interacting with our custom scrollbar
			this.y = e? this.y - e.deltaY : this.y
			this.y = clamp(this.y, 0, this.containerDiv.scrollHeight - window.innerHeight)
	}
}

• In our render loop (executed every frame), we update our curtains.js ScrollValues and container div scrollTop at exactly the same time, making use of the curtains.js lerp function to add smooth scroll

getDelta(){
	//this function helps us know how long it took to render our frame
	//helps keep lerping timing consistent regarless of frame rate
        let delta = (performance.now() - this.lastFrame) / 1000
        delta = delta > 1.0 ? 1.0 : delta
        this.lastFrame = performance.now()
        return delta
    }

onRender(){
	let delta = this.getDelta()

	this.scroll.value = this.curtains.lerp(this.scroll.value, this.y, delta * 2.5)
	// lerping adds a smooth scroll effect
	this.curtains.updateScrollValues(0, this.scroll.value)
	//we update the curtian.js scroll
	this.containerDiv.scrollTop = this.scroll.value
// we update our container div scrollposition at exactly the same rate as the curtains

• Using overflow:hidden means we need to implement a custom scrollbar for desktop

‍

And there we go, updating the the container div’s scrollTop inside the curtain.js render loop will completely eliminate all misalignment scroll jank.

‍

While smooth-scrolling and overriding default scrolling behavior is quite a controversial UX decision, in this use case, it was the best solution get the experience working smoothly across devices.