Building recommendation systems with mindsdb

Where we can use mindsdb?

MindsDB is an open-source platform that makes it incredibly easy to build AI applications. Want to create a feature that recommends news based on user interests? Traditionally, you'd need to build an inference layer, set up a complex model training pipeline, and manage datasets.

This process can be time-consuming and require specialized expertise. MindsDB changes all that. It provides an out-of-the-box solution for all these challenges, allowing you to build and deploy AI features quickly and effortlessly. Interesting, right? Let's explore how MindsDB can revolutionize your AI development.

How to run mindsdb locally?

• mindsdb/mindsdb:latest (or mindsdb/mindsdb) It is the lightweight Docker image of MindsDB that comes with these integrations preloaded.
• mindsdb/mindsdb:lightwood It is the Docker image of MindsDB that comes with these integrations and the Lightwood integration preloaded.
• mindsdb/mindsdb:huggingface It is the Docker image of MindsDB that comes with these integrations and the Hugging Face integration preloaded.

Use this docker compose file

services:
  postgres:
    image: postgres:latest # Use the latest PostgreSQL image
    container_name: mindsdb-postgres # Name the container
    environment:
      POSTGRES_PASSWORD: mobilelabs # Set the database password
    volumes:
      - pgdata:/var/lib/postgresql/data # Persist data with a named volume
    ports:
      - "5432:5432" # Expose the default PostgreSQL port

  mindsdb:
    image: mindsdb/mindsdb:latest # Use the latest MindsDB image
    container_name: mindsdb-instance # Name the container
    ports:
      - "47334:47334" # Expose the MindsDB API port
    command:
      - mindsdb start
      - --config
      - /config/mindsdb.config.json # Specify the MindsDB configuration file
    volumes:
      - ./mindsdb_config:/config # Mount the configuration directory
    depends_on:
      - postgres # Ensure PostgreSQL starts before MindsDB

volumes:
  pgdata: # Define the named volume for PostgreSQL data

networks: # Define a custom network
  mindsdb-net:
    driver: bridge

• mindsdb: mindsdb docker instance running in 127.0.0.1:47334
• postgres: postgres database which will be used as datasource for this tutorial

Open 127.0.0.1:47334 in browser

Create LightFM recommendation system using mindsdb

Let's understand LightFM model in detail, We need to build a news portal where news feeds are personalised based on user interest.

First we need to create new database in our postgres instance

docker exec -it mindsdb-postgres psql -U postgres -c "create database user_interactions;"

This will new database user_interactions in our instance

create database user_interactions;

Now let's add datasource in our mindsdb instance

Select New Datasource

Enter pgsql connection details, you can use any pgsql instance be it from local or any cloud like AWS RDS, Redshift, etc.

use the password mobilelabs which is mentioned in docker-compose.yml filed.

Make sure it passes the test connection and save and continue

Now we have successfully connected out datasource to mindsdb.

What is next?

Import the datas into datasets, we need to have user interests, news and users data to contiune.

Create users

CREATE TABLE user_interactions.users (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) NOT NULL
);

INSERT INTO user_interactions.users (name) VALUES
('Alice Wonderland'),
('Bob The Builder'),
('Charlie Chaplin'),
('Diana Prince'),
('Elon Musk');
('Vasanth Jagadeesan');

News data

CREATE TABLE user_interactions.news (
    id INT PRIMARY KEY,
    title VARCHAR(255) NOT NULL,
    url VARCHAR(255)
);

INSERT INTO user_interactions.
news (id, title, url) VALUES
(1, 'Y Combinator announces YC Continuity Fund', 'https://example.com/yc-continuity-fund'),
(2, 'Apple releases new MacBook Pro with Retina display', 'https://example.com/new-macbook-pro'),
(3, 'Google launches self-driving car project', 'https://example.com/self-driving-car'),
(4, 'Microsoft acquires GitHub for $7.5 billion', 'https://example.com/microsoft-github'),
(5, 'Amazon announces Prime Day 2024', 'https://example.com/prime-day-2024');

User interests

CREATE TABLE user_interactions.interests (
  id SERIAL PRIMARY KEY,
  user_id INT ,
  item_id INT,
  rating INT
);

INSERT INTO user_interactions.interests (user_id, item_id, rating) VALUES 
(1, 1, 1),
(1, 4, 1),
(3, 2, 1),
(2, 4, 1);

Now we have news, users and interests ready in our datasource.

Time to do magic with mindsdb

We need to create our LightFM ML Engine in mindsdb

CREATE ML_ENGINE lightfm
FROM lightfm;

Create a model in mindsdb

CREATE MODEL news_recommender
FROM user_interactions (SELECT * FROM interests)
PREDICT item_id
USING
  engine = 'lightfm',
  item_id = 'item_id',
  user_id = 'user_id',
  threshold = 4,
  n_recommendations = 10,
  evaluation = true;

• user_interactions is the db and interests is the table which holds user and news article interest martix.
• predict is prediction field form this table
• item_id is news article id in the interestes table
• user_id is user id in the interests table

Let's build the query that returns recommendation for the user based on the interest.

SELECT *
FROM user_interactions.news AS a
JOIN (SELECT b.*
FROM news_recommender AS b
where user_id = 3
using recommender_type = 'user_item') as b
ON a.id = b.item_id

Result

For more content please subscribe!