Leader Election with ZooKeeper

1. ZooKeeper Master-Election Implementation

1.1. pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.zsk</groupId>
    <artifactId>test_zk</artifactId>
    <version>1.0-SNAPSHOT</version>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.20.RELEASE</version>
    </parent>

    <dependencies>
        <!--zookeeper-->
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.5.6</version>
        </dependency>
        <!--test-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>1.5.20.RELEASE</version>
                <configuration>
                    <executable>true</executable>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

1.2. Code

public class MasterElection
{
    private static final String MATER_PATH = "/master/election";
    private String machineName;

    public MasterElection(String machineName)
    {
        this.machineName = machineName;
    }

    public static void main(String[] args) throws InterruptedException
    {
        IntStream.rangeClosed(1,5)//number sequence
                .mapToObj(index->"Machine" + index)//transform: add "Machine" in front
                .map(MasterElection::new)//transform: MasterElection constructor
                .map(task->(Runnable)()-> {//transform: create Runnable
                    try
                    {
                        task.go();
                    }
                    catch (Exception e)
                    {
                        e.printStackTrace();
                    }
                })
                .map(Thread::new)//transform: create Thread
                .forEach(Thread::start);//iterate and start

        TimeUnit.SECONDS.sleep(1000);
    }

    public ZooKeeper connect() throws Exception
    {
        // Execute asynchronously. Use CountDownLatch for synchronization and wait until the connection is created before continuing.
        CountDownLatch countDownLatch = new CountDownLatch(1);
        ZooKeeper zooKeeper = new ZooKeeper("127.0.0.1:2181", 5000, new Watcher()
        {
            @Override
            public void process(WatchedEvent watchedEvent)
            {
                countDownLatch.countDown();
            }
        });
        countDownLatch.await();
        System.out.println(machineName + " connected to ZooKeeper successfully");
        return zooKeeper;
    }

    public void go() throws Exception
    {
        // Execute asynchronously. Use CountDownLatch for synchronization and wait until the connection is created before continuing.
        ZooKeeper zooKeeper = connect();
        toBeMaster(zooKeeper);

    }

    private void toBeMaster(ZooKeeper zooKeeper)
    {
        zooKeeper.create(MATER_PATH, "".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL, new AsyncCallback.StringCallback()
        {
            @Override
            public void processResult(int rc, String path, Object ctx, String name)
            {
                if (rc == KeeperException.Code.OK.intValue())
                {
                    System.out.println(machineName + " created the node successfully and became master");
                    try
                    {
                        // Simulate business execution
                        TimeUnit.SECONDS.sleep(2);
                        // Simulate a crash
                        zooKeeper.delete(MATER_PATH, -1);
                        System.out.println(machineName + " crashed");
                    }
                    catch (Exception e)
                    {
                        e.printStackTrace();
                    }
                }
                else if (rc == KeeperException.Code.NODEEXISTS.intValue())
                {
                    System.out.println(machineName + " waiting");
                    try
                    {
                        zooKeeper.exists(MATER_PATH, new Watcher()
                        {
                            @Override
                            public void process(WatchedEvent event)
                            {
                                if (event.getType() == Event.EventType.NodeDeleted)
                                {
                                    toBeMaster(zooKeeper);
                                }
                            }
                        });
                    }
                    catch (Exception e)
                    {
                        e.printStackTrace();
                    }
                }
                else
                {
                    System.out.println(machineName + " abnormal state");
                }
            }
        }, "ctx_data");
    }
}

1.3. Test

First connect to ZooKeeper and create the /master node.

create /master test

1.4. Result

Machine3 connected to ZooKeeper successfully
Machine4 connected to ZooKeeper successfully
Machine2 connected to ZooKeeper successfully
Machine1 connected to ZooKeeper successfully
Machine5 connected to ZooKeeper successfully
Machine4 waiting
Machine1 created the node successfully and became master
Machine2 waiting
Machine3 waiting
Machine5 waiting
Machine1 crashed
Machine3 waiting
Machine2 created the node successfully and became master
Machine5 waiting
Machine4 waiting
Machine2 crashed
Machine4 waiting
Machine3 created the node successfully and became master
Machine5 waiting
Machine3 crashed
Machine5 waiting
Machine4 created the node successfully and became master
Machine4 crashed
Machine5 created the node successfully and became master
Machine5 crashed

Process finished with exit code -1

2. ZooKeeper Master-Election Principle

Like ZooKeeper Distributed Lock.md, create ephemeral sequential nodes + watch the deletion event of the node immediately before your own.

1. Each client creates an ephemeral sequential node under /master.
2. The client with the smallest number is elected master.
3. Other clients watch the deletion event of the node whose sequence is one smaller than their own. If it is deleted, the client checks whether it is now the smallest among all clients; if so, it is elected master.

It is similar to ZooKeeper Distributed Lock.md.