Machine Learning in Linux: Dalai – LLaMA and Alpaca

Dalai bills itself as “the simplest way to run LLaMA on your local machine”.

Given that our Machine Learning in Linux series focuses on apps that make it easy to experiment with machine learning, Dalai looks an interesting project to spotlight.

What’s LLaMA? It’s an acronym for Large Language Model Meta AI, a collection of open and efficient foundation language models designed to help researchers advance their work in this subfield of AI.

Large Languages Models trained on massive amount of text can perform new tasks from textual instructions. They can generate creative text, solve maths problems, answer reading comprehension questions, and much more. Like other large language models, LLaMA works by taking a sequence of words as an input and predicts a next word to recursively generate text.

Installation

Dalai needs Python <= 3.10 and Node.js >=18. On our Ubuntu 22.04 LTS test system, we need to replace our version of Node.js as it’s too old. The simplest way to upgrade on Ubuntu is via a PPA (personal package archive). Run the following command:

$ curl -fsSL https://deb.nodesource.com/setup_19.x | sudo -E bash - &&\
sudo apt-get install -y nodejs

We can verify the new version of node.js with the command:

$ node -v
v19.8.1

We can now proceed and use npx for the installation. On a fresh installation of Ubuntu 22.04 LTS we’ll also need to install npm, a package manager for Node.js and JavaScript.

$ sudo apt install npm

We’re now ready to install Dalai and its 7B model (we recommend you start with this model as it’s the smallest).

$ npx dalai llama install 7B

Part of the installation process quantizes the model. On a fairly modern processor the quantization process takes under a minute.

The web UI server is launched with the command:

$ npx dalai serve

Next page: Page 2 – In Operation and Summary

Pages in this article:
Page 1 – Introduction and Installation
Page 2 – In Operation and Summary