AI in the browser is here

Watch my 18 minute presentation of this topic with live demos here: https://youtu.be/6ic1-wtvYz4?si=SqXH9_IfNT7LAuR2.

This is what excites me about having AI run in the browser:

1. Client-side privacy. Customers are worried that their data is being trained/embedded into AI models, which may later get surfaced through some type of prompt hacking. There are lots of ways to prevent this, including keeping data and AI on the browser, where inference can happen without ever leaving the user’s computer.
2. Rich interaction. On every keystroke or click, you can run AI in the browser’s background to provide intelligent experiences.
3. Distributed computation. If the user’s computer has good enough resources, then offloading some of the computation to their browser instead of handling all that load on the servers could be a saver for you.
4. Accessibility. Javascript is easy to develop in, is the most popular programming language in the world, and when published on the web makes your AI experiences available for everyone.

AI can now be done in the browser, browser extensions, electron apps, and Node.js back-ends. Gone are the days where AI had to be written in Python.

Here are a few demos of different AI technologies that you can use directly on the front-end (and they’re available in Node.js for back-end development as well).

Tensorflow.JS

Official website: tensorflow.org

Notes

It can run in the browser using (in order from slowest to fastest):

• CPU
• WASM (web assembly)
• WebGL (100 times faster)

When running in Node.js, it utilizes C libraries, hardware accelaration, and Cuda where available.

Also, you can convert Python models to JavaScript.

Demos

• Webcam Pacman
• Weboji three.JS raccoon demo
• The Awesome Teachable Machine List
• TensorFlow.js demos
• tfjs/GALLERY.md
• Handtracking in Javascript
• Handsfree.js
• jammerjs

Web Speech API

Documentation: Using the Web Speech API

Notes

A machine-learning task that may seem so “mainstream” that it doesn’t feel like AI anymore is text to speech (TTS) and speech to text. This is supported now as a web API.

Demo

• Speech color changer

Transformers.JS

Official website: HuggingFace — Transformers.JS

Notes

Huggingface is THE hub right now for AI. They offer lots of things, including a Python and JavaScript library called “Transformers”, which uses models to accomplish a variety of tasks. The supported tasks at the moment are listed below:

Source: HuggingFace — Transformers.JS

You can use HuggingFace’s available models, or fine-tune a large language model, or convert any Python model to Javascript. Imagine a fine-tuned Llama-3 model running in your browser!

Transformers.JS downloads models to browser cache. Because it’s stored in local cache, it’s available on subsequent refreshes and offline.

Demos

• Transformers.JS examples
• Demo playground

LangChain.JS

Official website: LangChain.JS

Notes

I’m not yet on the LangChain craze that everyone else seems to be, but if you wanted to use it to interface with external large language models, this library is available in both Python and Javascript.

Gemini Nano Web API

Official website: Artificial Intelligence — Chrome for Developers

Notes

Google Chrome Canary, under a flag, downloads Gemini Nano embedded into Chrome itself. It’s only a matter of time before other browsers start shipping embedded large language models as well.

Demos

• Built-in AI Chat — a Hugging Face Space by Xenova
• Prompt API Playground

The Future

This article shows that AI is available in browsers now. Some models and tasks require heavy inference times. You can run these in web workers, or use other async tricks to make the lag seem ok. But, we’re going to want and expect more. That is why this web API proposal is interesting: WebGPU API. Additionally, AI models will continue to improve and quantization will help give us smaller models.

Additional reading

There are many other libraries that we haven’t covered that are available in Javascript, and I think they are worth exploring.

I just wanted to give a taste here of what is possible, and how easily it can be done.

Additional articles of interest may include:

• AI with JS: Beginner’s Guide
• Playing with Artificial Intelligence in the browser using TensorFlow
• An introduction to AI in Node.js — IBM Developer
• Unleashing the Power of JavaScript in Artificial Intelligence | HackerNoon
• Machine Learning in JavaScript
• How Hugging Face achieved a 2x performance boost for Question Answering with DistilBERT in Node.js — The TensorFlow Blog
• The Pros and Cons of Using JavaScript for Machine Learning
• Performance Comparison: JavaScript vs. Python for Machine Learning