1.9 Elasticsearch and MySQL Synchronization

1. Scheduled Synchronization

Use a scheduled task to pull data from MySQL page by page and write it into Elasticsearch. Added and modified data is straightforward: just reindex it. But what about deleted data? Add an extra update_time field. After the update finishes, delete everything with a value less than update_time.

2. cannel

Use cannel to parse the MySQL binlog and then send it to Kafka. Elasticsearch consumes Kafka. For add/update messages, reindex directly; for delete messages, delete the document.

3. kafka-connect

4. databus

5. References

• cannel-脉脉
• kafka-connect-脉脉
• databus-脉脉