Quick Start

Get up and running with ElasticRelay in 5 minutes

Quick Start

This guide will help you get ElasticRelay up and running in just a few minutes.

Prerequisites

Before you begin, make sure you have:

  • ElasticRelay installed (Installation Guide)
  • A running database (MySQL, PostgreSQL, or MongoDB)
  • A running Elasticsearch instance

Quick Start with MySQL

Step 1: Enable MySQL Binlog

First, ensure MySQL binary logging is enabled. Add to your my.cnf:

[mysqld]
server-id = 1
log_bin = mysql-bin
binlog_format = ROW
binlog_row_image = FULL

Restart MySQL:

sudo systemctl restart mysql

Step 2: Create a CDC User

Create a dedicated user for ElasticRelay:

CREATE USER 'elasticrelay'@'%' IDENTIFIED BY 'your_password';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'elasticrelay'@'%';
FLUSH PRIVILEGES;

Step 3: Create Configuration File

Create a config.json file:

{
  "version": "3.0",
  "data_sources": [
    {
      "id": "mysql-main",
      "type": "mysql",
      "host": "localhost",
      "port": 3306,
      "user": "elasticrelay",
      "password": "your_password",
      "database": "mydb",
      "tables": ["users", "orders"]
    }
  ],
  "elasticsearch": {
    "addresses": ["http://localhost:9200"]
  }
}

Step 4: Run ElasticRelay

Start ElasticRelay:

elasticrelay --config config.json

You should see output indicating that ElasticRelay is starting:

2024-12-08 10:00:00 INFO Starting ElasticRelay v3.0.0
2024-12-08 10:00:01 INFO Connected to MySQL: mysql-main
2024-12-08 10:00:02 INFO Connected to Elasticsearch
2024-12-08 10:00:03 INFO Starting snapshot for table: users
2024-12-08 10:00:05 INFO Snapshot complete, switching to CDC mode

Step 5: Verify Synchronization

Check that your data is in Elasticsearch:

curl -X GET "localhost:9200/mydb.users/_search?pretty"

Quick Start with PostgreSQL

Step 1: Enable Logical Replication

Add to your postgresql.conf:

wal_level = logical
max_replication_slots = 4
max_wal_senders = 4

Restart PostgreSQL:

sudo systemctl restart postgresql

Step 2: Create a CDC User

CREATE USER elasticrelay WITH REPLICATION PASSWORD 'your_password';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO elasticrelay;

Step 3: Create Configuration File

{
  "version": "3.0",
  "data_sources": [
    {
      "id": "postgres-main",
      "type": "postgresql",
      "host": "localhost",
      "port": 5432,
      "user": "elasticrelay",
      "password": "your_password",
      "database": "mydb",
      "schema": "public",
      "tables": ["users", "orders"]
    }
  ],
  "elasticsearch": {
    "addresses": ["http://localhost:9200"]
  }
}

Step 4: Run ElasticRelay

elasticrelay --config config.json

Quick Start with MongoDB

Step 1: Enable Replica Set

MongoDB change streams require a replica set. If you’re running standalone, convert it:

# Start MongoDB with replica set
mongod --replSet rs0

# Initialize replica set
mongosh --eval "rs.initiate()"

Step 2: Create a CDC User

use admin
db.createUser({
  user: "elasticrelay",
  pwd: "your_password",
  roles: [
    { role: "read", db: "mydb" },
    { role: "read", db: "local" }
  ]
})

Step 3: Create Configuration File

{
  "version": "3.0",
  "data_sources": [
    {
      "id": "mongodb-main",
      "type": "mongodb",
      "connection_string": "mongodb://elasticrelay:your_password@localhost:27017/?replicaSet=rs0",
      "database": "mydb",
      "collections": ["users", "orders"]
    }
  ],
  "elasticsearch": {
    "addresses": ["http://localhost:9200"]
  }
}

Step 4: Run ElasticRelay

elasticrelay --config config.json

Testing Your Setup

Insert Test Data

MySQL:

INSERT INTO users (id, name, email) VALUES (1, 'John Doe', 'john@example.com');

PostgreSQL:

INSERT INTO users (id, name, email) VALUES (1, 'John Doe', 'john@example.com');

MongoDB:

db.users.insertOne({ name: 'John Doe', email: 'john@example.com' })

Verify in Elasticsearch

# Search for the new document
curl -X GET "localhost:9200/mydb.users/_search?q=John&pretty"

You should see your data synchronized to Elasticsearch within seconds!

Common Issues

Connection Refused

If you see connection errors, verify:

  • Database is running and accessible
  • Elasticsearch is running and accessible
  • Credentials are correct
  • Firewall rules allow connections

No Data Syncing

If data isn’t syncing:

  • Check that binlog/logical replication is enabled
  • Verify the CDC user has correct permissions
  • Check ElasticRelay logs for errors
  • Ensure tables/collections are specified in config

What’s Next?

Now that you have ElasticRelay running, learn more about: