We recently hit a bottleneck in a visualization application that we are working on, specifically in the backed of the visualization app. The app consist of an ASP .NET Core backend, a CosmosDB for hot storage, Blob storage for cold storage and a React frontend to visualize both the hot and cold data.

We have the non-functional requirement (NFR) that data must be visible within 5 seconds (easy to test, hard to achieve). Our backend collects the data both from hot and cold storage and exposes this for the React fronted through a REST API. Works fine, but the NFR of 5 seconds are not achievable using this approach since the frontend will “hang” / “freeze” with a spinner until data is ready, which can easily take 20-30 seconds.

I had the idea of streaming data from the backend to the frontend and let the frontend do the processing of the data instead of the back end. While it would still take 20-30 seconds before the data was 100% visible in the frontend, we could at-least achieve the 5 second NFR since we are able to show the data as it is ready, which would greatly enhance the user experience.

The team was not fond of my idea, due to “JavaScript not being fast enough to process the data”. While this might have been true in the nineties, it is no longer true with the V8 engine powering the JavaScript of today. I decided to convince the team through a quick experimental benchmark where I would Implement Bubble Sort in C#/.NET Core and JavaScript/Node and compare the results. Oh, and just for the kicks, I did an implementation in C++ using Clang as compiler.

The source-code of the benchmark can be found on my Github: BubblesortBenchmark

For the benchmark, 50.000 elements are generated and bubble-sorted 10 times to avoid measuring “cold start”.

For the C++ benchmark, the clang compiler is used with optimization and without the -fsanitize=address flag. This flag will introduce bounds checking to make the C++ runtime comparable to that of C# and Node. The -O flag is also used to optimize the compiled code. Two bubble sort implementations have been done for C++, one using std:vector and another using arrays.

For the C# benchmark, DotNet Core 3.1 have been used with -configuration=Release flag to produce a optimized binary. Also for C#, two implementations have been one, one using List and another using a C# array.

For the JavaScript benchmark, a single implementation has been done using JavaScript arrays. Node v12 is used as runtime.

Now for the results!
Taken directly from the shell:

> clang -O -fsanitize=address main.cpp && a.exe 
Creating library a.lib and object a.exp 
30156ms to sort 50000 elements 10 times (Vector) 
22360ms to sort 50000 elements 10 times (Array)

> clang -O main.cpp && a.exe 
9984ms to sort 50000 elements 10 times (Vector) 
8859ms to sort 50000 elements 10 times (Array)

> dotnet run --configuration=Release 
28766ms to sort 50000 elements 10 times (List) 
14687ms to sort 50000 elements 10 times (Array)

> node index.js 
29131ms to sort 50000 elements 10 times

And visualized as bar graphs (lower is better):

The results are not that surprising.

C++ without address sanitizer outperforms all other implementations. The reason is simple, the program does not check for out of bounds when accessing the array and will therefore save one or more instructions per array access, which for Bubblesort is O(n^2). std:vector and array are also comparable in performance since very little overhead is introduced with the std:vector abstraction.

C# .NET Core using List is nearly two times slower than C# .NET using arrays. This could be attributed to the fact that an array of Integers in .NET is guaranteed to be a continuous set of integers where a List boxes the Integers into objects introducing additional indirect access.

JavaScripts approach to arrays are similar to that of C# lists, which is also evident in the benchmark. Comparing the C# List and JavaScript Array implementations, they are nearly 100% identical! Compared this with the C++ std:vector implementation using address sanitize, all three “list like” implementations performs the same.

One surprising aspect is that C# .NET Core array outperforms all C++ implementations when guarded with the address sanitize flag. I believe this is due to the just-in-time compilation nature of .NET Core, since .NET might deduce that no range-checks are needed during the JIT compilation process and thus save several instructions.

All in all, a nice little benchmark that has convinced the team that we can surely due data processing using JavaScript if needed.

Next step could be, to look into the JavaScript implementation and see if it can be improved by using e.g. Object.seal to increase performance. Another aspect could be to introduce a similar implementation in Rust which I would expect performed the same as the C++ arrays implementation using address sanitizer.

I am a huge fan of static web-sites with no fuss, especially in regards to blogs where ease of consuming information is the key. Recently, I have been looking into site generator frameworks to help generate fast and ‘no fuss sites’. One of my consultant buddies recommended me to look into ‘Gatsbyjs’.

So I did! Gatsbyjs is a site generator for React. It provides the ability to generate HTML based upon one or more React templates. What makes Gatsbyjs a bit special, compared to some of the other site generators that I have seen, is the fact that it abstracts away the file system (and other data sources) using a GraphQL.

GraphQL is a query language for API’s. Using GraphQL, one can request what is needed, including references to other resources, by specifying a query containing the types of resources and their relations to other resources. Compared to a REST API, which typically returns a predefined representation, GraphQL allows much more control and is type-safe since the resources are described as types (including references to other types) and not as endpoints .

Gatsby provides the ability to source data from many different sources: file-system, sql, mongodb, rest, etc. into GraphQL. In addition to being sourced, data can also be transformed, e.g. Markdown can be parsed for easier consumption, images can be optimized for the web, etc..

Pages can be generated in many ways. Simplest way is to put a React component into src/pages, which in turn is rendered to HTML and copied to output. Pages can also be generated programmatically, using the createPages API, or by importing a plugin.

Pages can run GraphQL queries that return data previously sourced, allowing the page to be populated with data from the GraphQL server and in turn statically generated and saved in one or more HTML files.

Gatsby also supports Reacts concept of “hydrate”, making it possible to add client-side React to a static generated page and thus provide app like functionality when all JS files have been loaded.

Next step for me is to try to implement this Blog using Gatsbyjs and see how it performs and manages - and if the results are good! Move my blog 100% to Gatsbyjs.

I have always been a big fan of THREE.js, a 3D JavaScript library that abstract away some of the complexities of OpenGL. Recently, I tried another library, Babylon.js, written in TypeScript and ofcourse for the browser.

I found Babylon to be on par with THREE on all the areas I needed, except for Camera control where Babylon really shines with it’s built in support for many different types.

Looking at npmjs.com, it’s clear that THREE today the ‘go to library’ when doing 3D in the browser. Currently THREE has 304k downloads a week, while Babylon.js has less than 6k of downloads a week, clearly THREE is more popular.

Size wise, I have not done any test on the produced bundle. I find this absurd in today’s world, where a website does a million AJAX request to load commercials anyway…

The only reason I recommend THREE over Babylon.js is because of it’s popularity on npm. Else, go with any library - they both solve the three-dimensional problem equally well in my opinion.

I recently had to debug an issue where the browser redirected the user. Debugging this was a pain, since the browser would clear all my views in developer console whenever the redirect happened.

I thought there must be a better way and yes! a bit of googling and I found this Gem:

window.addEventListener("beforeunload", function() { debugger; }, false)

This will break whenever the beforeunload event is executing, which happens right before a redirect.

Simply copy and paste into the console, and you are good to go!

This allowed me to see the exact call-stack leading to the beforeunload event.

In my concrete case the issue was related to a Cookie not being set due to SameSite not being set in a Cookie, which is a requirement by Chrome since version 80.

Introducing automation is something I am extremely keen on. Recently, I have been using Cypress at work, an end-to-end testing framework for browser based applications.

I got the idea that Cypress could be used to do Robot Process Automation (RPA) through the use of GitHub actions. My idea was to:

• Implement a Cypress test suite which opened a specific website.
• Traversed the website, to find meaningful data.
• Email the data.
• Let GitHub actions start the process.

I look at the bond prices every day on Nasdaq to get an indication on which direction my mortgage is going. A good candidate to automate: let Cypress look at the mortage and send me the result on email such that:

• I do not have to remember to check every day
• I do not have to access the site manually and spend time on this.
• I get the result every day at a specific time, e.g. 10:00..

Implementation is super simple, write a test that opens the specific Nasdaq page, selects the element of interest and then emails the element of interest to me using an email client. For my tests, I used mail-slurp since it is free and works well.

The complete cypress code is here:

/// <reference types="cypress" />
import { MailSlurp } from "mailslurp-client";

let last = undefined;
const TO = Cypress.env("TO");
const API\_KEY = Cypress.env("API\_KEY");
const mailslurp = new MailSlurp({apiKey:API\_KEY});
const from = Cypress.env("FROM");

context('Actions', () => {
    it("Open Nasdaq", ()=>
    {
        cy.log(MailSlurp);
        cy.visit('http://www.nasdaqomxnordic.com/bonds/denmark/microsite?Instrument=XCSE0%3A5NYK01EA50');
    });

    it("Find Last", ()=>
    {
        cy.get(".db-a-lsp").should((e)=>
        {
            if (e != null)
            {
                last = e.first().text();
            }
        });
    });

    it("Email Last", async ()=>
    {
        //cy.log(last);
        await mailslurp.sendEmail(from,
            {
                to:\[TO\],
                subject:'0.5% kurs ' + last,
                body:last
            }
        ); 
    });
});

Quick and dirty. Three actions are defined:

• Open Nasdaq: simply opens the URL in question.
• Find Last: finds the price of the latest sale of the given bond.
• Email Last: emails me the result

Simple as that. Running the “test” does exactly that. To automate, I use GitHub actions to start Cypress on every push but also every day at 09:00 UTC time.

I will definitely try to identify more use cases where this can be applicable in the future.

I am a frequent WebSocket user. WebSocket is perhaps the single-best thing that has happened to web development since the introduction of HTML5, due to its bi-directional and ‘real-time’ characteristics.

I have frequently used a websocket conncetion to keep the client up to date with regards to changes from the server, especially when doing game related development.

Recently, I stumbled upon SignalR which is more or less an RPC library for .NET and JavaScript. It provides the ability for a JavaScript application running in the browser to invoke functions directly on the .NET server application, and vice versa.

The transport channel is simply a websocket connection, thus anywhere websockets are supported SignalR should in theory be supported. In addition, SignalR is 100% open - allowing anyone the possibility to implement a SignalR client or server for any other language.

I have yet to tryout the library myself - but it is definitely on my to do list!

I frequently need to share typed structures between applications, such as between an ASP.NET C# Backend and an Typescript React single page application.

For this, I have used Google’s protobuf, a binary serialization library.

In protobuf, one defines the intended information in .proto files and uses a Protobuf compiler to auto generate language specific source files.

syntax = “proto3”; message Position { float x = 1; float y = 2; }

Here I define a message called Position, containing two simples float values: x and y. 1 and 2 denotes the order of which the fields appear during the serialization process - for backward compatibility reasons changing the name is allowed but changing the numbers are not since this will change the order on the wire!

Using a Protobuf compiler, such as protoc, language specific versions of the messages can be generated to fit the required language. For C#, the message above results in a class called Position with public Properties called X and Y, including several serialization methods and Properties such as WriteTo, MergeFrom and Parser.

For the web, protobufjs exists, which is a protobuf compiler implemented in JavaScript. This compiler can generate JavaScript and TypeScript source files to be consumed both in node.js and in the browser. I typically generate files for node.js and use a bundler such as parcel-bundler or webpack to bundle for the browser.

This provides binary transport of my messages from backend to frontend, with type security - very nice indeed.

Lately, I have looked into Apache Thrift, which is very similar to Protobuf but with support for many more programming languages out of the box, and with the added bonus of also supporting Remote Procedure Calls (RPC) . (although RPC can be achieved in Protobuf using gRPC)

The syntax of Thrift messages are very similar to Protobuf, the same message above would be written as:

struct Position { 1: double x, 2: double y }

Note: floats are not supported by Thrift, only doubles.

Looking at the supported languages, it seems to dwarf that of protoc (the default compiler of Protobuf) - with support for nearly 30 languages of the the box. Very nice indeed.

C# and TypeScript is also supported, making the Thrift a prime candidate for my typical needs. However, diving a bit deeper into what support means, it quickly shows that not all languages are equally supported, this includes TypeScript.

Generating browser compatible TypeScript that supports binary serialization is not supported. Node.js compatible TypeScript can be generated that supports binary serialization, but this generated code cannot be consumed by bundlers such as WebPack or Parcel Bundler. RPC is also not supported in the browser, although not required for the project in which I decided to test out Thrift.

Looking through NPM, I found “browser-thrift” - a patched version of node thrift that can be consumed by bundlers, only requiring that require(‘thrift’) is replaced with require(‘browser-thrift’) in generated thrift code.

After a few tries, I never got this approach to work and reverted back to using Protobuf.

Summary: Binary serialization is not natively supported by Thrift in the browser - making Protobuf the obvious choice until this is implemented in Thrift.

I was looking for a fast JavaScript vector library and found glMatrix. glMatrix is a Matrix and Vector library with high performance. The high performance is achieved using API conventions, e.g. by avoiding reducing use of implicit memory allocation and by carefully designing the usage of the library.

The lib does not feel natural but I do like that I know that memory management performance will not explode in my face when using it.

Creating a 2D vector is done explicitly by writing

let v = vec2.create();

while adding vectors together is done by

vec2.add(out, v1, v2);

operations such as add, sub, etc. do not enfer any memory allocation what so ever, making the operations fast and without garbage collection at a later time.

The API does feel very ‘C’ like. I really miss the ability of .NET where one can allocate to the stack by using Structs.

In Photoshop CS6+ it is possible to do scripting using JavaScript. Photoshop exposes a DOM, similar to that found on the web, where one can manipulate layers, etc.

Creating a new document is as simple as invoking the following code:

var docRef = app.documents.add(2,4);

Next up, we can add a text layer:

var artLayerRef = docRef.artLayers.add(); artLayerRef.kind = LayerKind.TEXT;

and set the text of the text layer:

var textItemRef = artLayerRef.textItem; textItemRef.contents = “Hello world”;

Simple as that.

I had an idea that I might be able to use Photoshop for some mockup purposes using scripting. Not yet verified though.

I needed a polygon library for my latest fun project (a TypeScript Build Engine). Google and NPM showed me the way to PolyK.JS, a very simple Polygon library that can do calculations on simple polygons, including Convex ones.

To my suprise, the library includes TypeScript  definitions making the lib super easy to consume for my application. Supported operations include:

• GetArea Gets the area of the polygon.
• Triangulate Triangulates the simple polygon
• Slice Slices the polygon.
• Raycast Finds the closest point on the polygon given the intersection of a ray.
• ClosestEdge Finds the closest edge of the polygon.
• ContainsPoint Checks if a given point is within the polygon.

The library assumes that the polygon’s vertices are defined in an number[] array of X,Y coordinates. Super easy to use and highly recommended!

In the past I have used D3.js and even Highchart.js for my charting needs. Today, I decided to tryout yet another Open Source alternative: Chart.js.

Chart.js is available through npm and @typings also exists for TypeScript fans such as myself:

npm install chart.js —save

npm install @typings/chart.js —save

Chart.js can be included with the script tag, CommonJs, ES6 and RequireJS. Although for HTML5 programming I always do the legacy route using script tag (I know.. I need to move with the tech).

Chart rendering happens in a canvas, nothing fancy DOM manipulation which I really like, especially when working with ReactJS. The chart look and feel is very generic, nothing fancy here.

I really like the tool-tips though:

Chart.js is also responsive, thus scaling to different screen sizes. I have yet to tryout this feature though. Apart from this, it supports animation, multiple axis, mixed chart types, and more.

8 chart types are provided out of the box. Definitely recommended.

The latest feature request required the ability to export the data contained within an HTML table to a file on the disk. To do this, I looked at the W3C Writer API: https://www.w3.org/TR/file-writer-api/

This API is not yet implemented in all browsers, and I thought that I had to look elsewhere. But then I found this: https://github.com/eligrey/FileSaver.js/ an implementation of the W3C standard usable by modern browsers.

Including FileSaver.js into my document allows me to call saveAs as if my browser natively supported. Saving some text to disk is as simple as making a Blob with the text content, correct mime-type and then calling saveAs:

let blob = new Blob("text", 
{
 type:"text/plain;charset=utf-8"
});
saveAs(blob, "exported\_text.txt");

Using this in TypeScript requires the correct typings for saveAs. I didn’t find them for this project, I simply declared saveAs as any type:

declare var saveAs;

Super easy, and verified to work on my PC using latest Chrome and IE :-)

Last week I started a prototype gamedev project where players can join a game using the mobile phones but only see the action on a shared screen. Think hot-seat where the controllers are the mobile phones.

I decided that I wanted to write an 2D HTML5 game in the browser using Canvas. But before utilizing my HTML5 Canvas skills I looked at what libraries might be able to help me in the endeavor.

It turns out that there are alot of libraries that can take care of basic sprite and tile rendering. PixiJS is such a library, providing a nice deferred renderer using hierarchical stage abstraction.

What I really fell in love with regarding PixiJS is the fact that it both support WebGL and Canvas rendering. This means that if the browser supports WebGL, it will utilize WebGL or else it will fallback to Canvas - nice, although not sure how this translates in a real world scenario where the Canvas typical is so much slower compared to the hardware accelerated rendering provided by OpenGL.

Rendering stuff with PixiJS is done by setting up some container in which one can put DisplayObjects:

let stage = new PIXI.Container();

Sprites can be created and added to the container:

let launcherTex = PIXI.Texture.fromImage('images/sprites/launcher.png');
let sprite = new PIXI.Sprite(launcherTex);
sprite.x = 123;
sprite.y = 123;
sprite.anchor.x = 0.5;
sprite.anchor.y = 0.5;

stage.addChild(sprite);

When done setting up the stage object, it can be send to the renderer for rendering:

renderer.render(stage);

PIXI.Containers are also DisplayObjects, and can be added to other containers, e.g. this is possible:

let stage1 = new PIXI.Container();
let stage2 = new PIXI.Container();

let final = new PIXI.Container();
final.addChild(stage1);
final.addChild(stage2);

renderer.render(final);

A Sprite is just a Container and can contain other sprites/other Display-objects, making it possible to hierarchical subdivide a scene into smaller and smaller components.

The GIF shows my prototype in action using PixiJS.

Scorched Earth / Worms inspired but currently with a lack of terrain destruction and pretty graphics :-)