Setting Up a Local AI-Powered Ruby on Rails Environment

Overview

Integrating AI into your Ruby on Rails application can enhance its capabilities, but relying on external APIs can be costly. This guide will show you how to set up a local AI environment using llamafile and Foreman to run your AI models alongside your Rails application.

Step 1. Download llamafile

First, download the Meta-Llama-3-8B-Instruct.Q4_0.llamafile or a model of your choice from huggingface.co and save it in the storage directory within your Rails application.

cd storage
wget https://huggingface.co/Mozilla/Meta-Llama-3-8B-Instruct-llamafile/resolve/main/Meta-Llama-3-8B-Instruct.Q4_0.llamafile?download=true

Make sure the downloaded llamafile is executable:

chmod +x storage/Meta-Llama-3-8B-Instruct.Q4_0.llamafile

Step 2. Set Up Foreman

Install Foreman, a tool to manage multiple processes for your Rails application:

gem install foreman

Create or update the Procfile.dev in your Rails application to include the web server and AI model:

# Procfile.dev
web: bin/rails server
ai: storage/Meta-Llama-3-8B-Instruct.Q4_0.llamafile --nobrowser

Step 3. Start Your Rails Application

Run your Rails application along with the AI model using Foreman:

foreman start -f Procfile.dev

Note: The llamafile readme provides addional gotchas and tips for running the server locally.

Step 4. Happy Coding and AI Integrating!

Your AI model will be available at http://localhost:8080, providing an API similar to OpenAI’s. You can use the OpenAI Gem by configuring it to point to your local llamafile API, or develop your own API client.
For more details on the API endpoints, refer to the llamafile repository.