Machine Learning in Linux: Dalai – LLaMA and Alpaca

In Operation

Point our web browser to http://localhost:3000

Here’s example output to a question we posed.

The dashboard lets you customize the configuration options. For example, we can increase the number of response words by changing n_predict. The project doesn’t explain the parameters in any detail.

A number of templates are available from a dropdown box.

• ai-dialog – user interacts with AI. AI is helpful, kind, obedient, honest, and knows its own limits.
• chatbot – an intelligent chatbot.
• default – writes a response that appropriately completes the request.
• instruction-alpaca
• rewrite-alpaca
• translate – translates English text to a foreign language.
• tweet-sentiment

Summary

Dalai lets you run ChatGPT-like models locally on your device with a simple installation although we had to update Node.js from the standard package available in Ubuntu 22.04 LTS.

From our initial testing, we find the 13B and 30B LLaMA model to offer a more natural conversation than ChatGPT. However, we need to perform a lot more testing and experiment with the other models and investigate Alpaca.

One major criticism of all LLMs is that because they are “large”, they need huge globs of computational power. Bear in mind you’ll also need a lot of disk space to store Dalai’s models. The smallest model (7B) consumes over 31GB hard disk (quantizing reduces this to approximately 4GB). The largest model uses a whopping 432GB of disk space (quantized 41GB). And don’t forget that the larger models have high RAM requirements.

Dalai is cross-platform software running under Linux as well as macOS and Windows.

At the time of writing the latest version of Dalai is 0.3.1. We can update to that version with the command:

$ npx dalai@0.3.1 setup

The web UI is powered by llama.cpp, llama-dl CDN and alpaca.cpp. Note, llama-dl’s repository is currently disabled due to a DMCA takedown notice.

Website: cocktailpeanut.github.io/dalai
Support: GitHub Code Repository
Developer: cocktailpeanut and contributors
License: MIT License

For other useful open source apps that use machine learning/deep learning, we’ve compiled this roundup.

Dalai is written in CSS and JavaScript. Learn CSS with our recommended free books and free tutorials. Learn JavaScript with our recommended free books and free tutorials.

Pages in this article:
Page 1 – Introduction and Installation
Page 2 – In Operation and Summary