RPC Server for Erlang, In Java

We are using Erlang to do some serious things, one of them is indeed part of a banking system. Erlang is a perfect language in concurrent and syntax (yes, I like its syntax), but lacks static typing (I hope new added -spec and -type attributes may be a bit helping), and, is not suitable for processing massive data (performance, memory etc). I tried parsing a 10M size XML file with xmerl, the lib for XML in OTP/Erlang, which causes terrible memory disk-swap and I can never get the parsed tree out.

It's really a need to get some massive data processed in other languages, for example, C, Java etc. That's why I tried to write RPC server for Erlang, in Java.

There is a jinterface lib with OTP/Erlang, which is for communication between Erlang and Java. And there are docs for how to get it to work. But, for a RPC server that is called from Erlang, there are still some tips for real world:

1. When you send back the result to caller, you need set the result as a tuple, with caller's tag Ref as the first element, and the destination should be the caller's Pid. It's something like:

OtpErlangTuple msg = new OtpErlangTuple(new OtpErlangObject[] {call.tag, tResult});
sConnection.send(call.to, msg); 

where, call.tag is a OtpErlangRef, and tResult can be any OtpErlangObject, call.to is a OtpErlangPid.

2. If you need to send back a massive data back to caller, the default buffer size of OtpErlangOutputStream is not good, I set it to 1024 * 1024 * 10

3. Since there may be a lot of concurrent callers call your RPC server, you have to consider the concurrent performance of your server, I choose using thread pool here.

The RPC server in Java has two class, RpcNode.java, and RpcMsg.java:

package net.lightpole.rpcnode;

import com.ericsson.otp.erlang.OtpErlangAtom;
import com.ericsson.otp.erlang.OtpErlangList;
import com.ericsson.otp.erlang.OtpErlangObject;
import com.ericsson.otp.erlang.OtpErlangPid;
import com.ericsson.otp.erlang.OtpErlangRef;
import com.ericsson.otp.erlang.OtpErlangTuple;

/**
 *
 * @author Caoyuan Deng
 */
public class RpcMsg {

    public OtpErlangAtom call;
    public OtpErlangAtom mod;
    public OtpErlangAtom fun;
    public OtpErlangList args;
    public OtpErlangPid user;
    public OtpErlangPid to;
    public OtpErlangRef tag;

    public RpcMsg(OtpErlangTuple from, OtpErlangTuple request) throws IllegalArgumentException {
        if (request.arity() != 5) {
            throw new IllegalArgumentException("Not a rpc call");
        }

        /* {call, Mod, Fun, Args, userPid} */
        if (request.elementAt(0) instanceof OtpErlangAtom && ((OtpErlangAtom) request.elementAt(0)).atomValue().equals("call") &&
                request.elementAt(1) instanceof OtpErlangAtom &&
                request.elementAt(2) instanceof OtpErlangAtom &&
                request.elementAt(3) instanceof OtpErlangList &&
                request.elementAt(4) instanceof OtpErlangPid &&
                from.elementAt(0) instanceof OtpErlangPid &&
                from.elementAt(1) instanceof OtpErlangRef) {

            call = (OtpErlangAtom) request.elementAt(0);
            mod = (OtpErlangAtom) request.elementAt(1);
            fun = (OtpErlangAtom) request.elementAt(2);
            args = (OtpErlangList) request.elementAt(3);
            user = (OtpErlangPid) request.elementAt(4);
            to = (OtpErlangPid) from.elementAt(0);
            tag = (OtpErlangRef) from.elementAt(1);

        } else {
            throw new IllegalArgumentException("Not a rpc call.");
        }
    }

    /* {'$gen_call', {To, Tag}, {call, Mod, Fun, Args, User}} */
    public static RpcMsg tryToResolveRcpCall(OtpErlangObject msg) {
        if (msg instanceof OtpErlangTuple) {
            OtpErlangTuple tMsg = (OtpErlangTuple) msg;
            if (tMsg.arity() == 3) {
                OtpErlangObject[] o = tMsg.elements();
                if (o[0] instanceof OtpErlangAtom && ((OtpErlangAtom) o[0]).atomValue().equals("$gen_call") &&
                        o[1] instanceof OtpErlangTuple && ((OtpErlangTuple) o[1]).arity() == 2 &&
                        o[2] instanceof OtpErlangTuple && ((OtpErlangTuple) o[2]).arity() == 5) {
                    OtpErlangTuple from = (OtpErlangTuple) o[1];
                    OtpErlangTuple request = (OtpErlangTuple) o[2];

                    try {
                        return new RpcMsg(from, request);
                    } catch (IllegalArgumentException ex) {
                        ex.printStackTrace();
                    }
                }
            }
        }
        
        return null;
    }
}

package net.lightpole.rpcnode;

import com.ericsson.otp.erlang.OtpAuthException;
import com.ericsson.otp.erlang.OtpConnection;
import com.ericsson.otp.erlang.OtpErlangAtom;
import com.ericsson.otp.erlang.OtpErlangExit;
import com.ericsson.otp.erlang.OtpErlangObject;
import com.ericsson.otp.erlang.OtpErlangString;
import com.ericsson.otp.erlang.OtpErlangTuple;
import com.ericsson.otp.erlang.OtpSelf;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * Usage:
 *   $ erl -sname clientnode -setcookie mycookie
 *   (clientnode@cmac)> rpc:call(xnodename@cmac, 'System', currentTimeMillis, []).
 * 
 * @author Caoyuan Deng
 */
public abstract class RpcNode {

    public static final OtpErlangAtom OK = new OtpErlangAtom("ok");
    public static final OtpErlangAtom ERROR = new OtpErlangAtom("error");
    public static final OtpErlangAtom STOPED = new OtpErlangAtom("stoped");
    private static final int THREAD_POOL_SIZE = 100;
    private OtpSelf xSelf;
    private OtpConnection sConnection;
    private ExecutorService execService;

    public RpcNode(String xnodeName, String cookie) {
        this(xnodeName, cookie, THREAD_POOL_SIZE);
    }

    public RpcNode(String xnodeName, String cookie, int threadPoolSize) {
        execService = Executors.newFixedThreadPool(threadPoolSize);

        startServerConnection(xnodeName, cookie);
        loop();
    }

    private void startServerConnection(String xnodeName, String cookie) {
        try {
            xSelf = new OtpSelf(xnodeName, cookie);
            boolean registered = xSelf.publishPort();
            if (registered) {
                System.out.println(xSelf.node() + " is ready.");
                /**
                 * Accept an incoming connection from a remote node. A call to this
                 * method will block until an incoming connection is at least
                 * attempted.
                 */
                sConnection = xSelf.accept();
            } else {
                System.out.println("There should be an epmd running, start an epmd by running 'erl'.");
            }
        } catch (IOException ex) {
            Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
        } catch (OtpAuthException ex) {
            Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    private void loop() {
        while (true) {
            try {
                final int[] flag = {0};

                final OtpErlangTuple msg = (OtpErlangTuple) sConnection.receive();

                Runnable task = new Runnable() {

                    public void run() {
                        RpcMsg call = RpcMsg.tryToResolveRcpCall(msg);

                        if (call != null) {
                            long t0 = System.currentTimeMillis();

                            flag[0] = processRpcCall(call);

                            System.out.println("Rpc time: " + (System.currentTimeMillis() - t0) / 1000.0);
                        } else {
                            try {
                                sConnection.send(sConnection.peer().node(), new OtpErlangString("unknown request"));
                            } catch (IOException ex) {
                                Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
                            }
                        }
                    }
                };

                execService.execute(task);

                if (flag[0] == -1) {
                    System.out.println("Exited");
                    break;
                }

            } catch (OtpErlangExit ex) {
                Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
            } catch (IOException ex) {
                Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
            } catch (OtpAuthException ex) {
                Logger.getLogger(RpcNode.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
    }

    protected void sendRpcResult(RpcMsg call, OtpErlangAtom head, OtpErlangObject result) throws IOException {
        OtpErlangTuple tResult = new OtpErlangTuple(new OtpErlangObject[] {head, result});

        // Should specify call.tag here
        OtpErlangTuple msg = new OtpErlangTuple(new OtpErlangObject[]{call.tag, tResult});
        // Should specify call.to here
        sConnection.send(call.to, msg, 1024 * 1024 * 10); 
    }

    public abstract int processRpcCall(RpcMsg call);
    

    // ------ helper
    public static String getShortLocalHost() {
        return getLocalHost(false);
    }

    public static String getLongLocalHost() {
        return getLocalHost(true);
    }

    private static String getLocalHost(boolean longName) {
        String localHost;
        try {
            localHost = InetAddress.getLocalHost().getHostName();
            if (!longName) {
                /* Make sure it's a short name, i.e. strip of everything after first '.' */
                int dot = localHost.indexOf(".");
                if (dot != -1) {
                    localHost = localHost.substring(0, dot);
                }
            }
        } catch (UnknownHostException e) {
            localHost = "localhost";
        }

        return localHost;
    }
}

As you can see, the RpcNode is an abstract class, by implement int processRpcCall(RpcMsg call), you can get your what ever wanted features. For example:

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package net.lightpole.xmlnode;

import basexnode.Main;
import com.ericsson.otp.erlang.OtpErlangAtom;
import com.ericsson.otp.erlang.OtpErlangList;
import com.ericsson.otp.erlang.OtpErlangObject;
import com.ericsson.otp.erlang.OtpErlangString;
import java.io.IOException;
import net.lightpole.rpcnode.RpcMsg;
import net.lightpole.rpcnode.RpcNode;

/**
 *
 * @author dcaoyuan
 */
public class MyNode extends RpcNode {

    public MyNode(String xnodeName, String cookie, int threadPoolSize) {
        super(xnodeName, cookie, threadPoolSize);
    }

    @Override
    public int processRpcCall(RpcMsg call) {
        final String modStr = call.mod.atomValue();
        final String funStr = call.fun.atomValue();
        final OtpErlangList args = call.args;

        try {
            OtpErlangAtom head = ERROR;
            OtpErlangObject result = null;

            if (modStr.equals("xnode") && funStr.equals("stop")) {
                head = OK;
                sendRpcResult(call, head, STOPED);
                return -1;
            }

            if (modStr.equals("System") && funStr.equals("currentTimeMillis")) {
                head = OK;
                long t = System.currentTimeMillis();
                result = new OtpErlangLong(t);
            } else {
                result = new OtpErlangString("{undef,{" + modStr + "," + funStr + "}}");
            }

            if (result == null) {
                result = new OtpErlangAtom("undefined");
            }

            sendRpcResult(call, head, result);
        } catch (IOException ex) {
            ex.printStackTrace();
        } catch (Exception ex) {
        }

        return 0;
    }
}

I tested MyNode by:

$ erl -sname clientnode -setcookie mycookie
...
(clientnode@cmac)> rpc:call(xnodename@cmac, 'System', currentTimeMillis, []).

And you can try to test its concurrent performance by:

%% $ erl -sname clientnode -setcookie mycookie
%% > xnode_test:test(10000)

-module(xnode_test).

-export([test/1]).

test(ProcN) ->
    Workers = [spawn_worker(self(), fun rpc_parse/1, {})        
     	       || I <- lists:seq(0, ProcN - 1)],
    Results = [wait_result(Worker) || Worker <- Workers].

rpc_parse({}) ->
    rpc:call(xnodename@cmac, 'System', currentTimeMillis, []).

spawn_worker(Parent, F, A) ->
    erlang:spawn_monitor(fun() -> Parent ! {self(), F(A)} end).

wait_result({Pid, Ref}) ->
    receive
        {'DOWN', Ref, _, _, normal} -> receive {Pid, Result} -> Result end;
        {'DOWN', Ref, _, _, Reason} -> exit(Reason)
    end.

I spawned 10000 calls to it, and it run smoothly.

I'm also considering to write a more general-purpose RPC server in Java, which can dynamically call any existed methods of Java class.

Comments

1. kenneth -- 2008-10-11 16:00

Regarding the speed for Erlang when it comes to as you say massive datahandling I think you are a bit unprecise and unfair. In particular your example with XMERL and XML parsing is misleading. XMERL is an application written in Erlang, we know that it is very slow and memory hungry today and we also know why. A new version is on it's way and it will have much better performance and will need significantly less memory as well. There is also another XML parser called ErlSOM which is much paster than the current XMERL. I think that calling Java from Erlang will note give better speed for massive data handling in the general case. It depends very much on what the datahandling is and how well that application performs (in Java or ERlang). And the inlcuding the overhead of the RPC call I think it in many cases pays off to stay on the Erlang side.

2. Caoyuan -- 2008-10-11 16:00

Hi kenneth,

Thanks for the note of new coming xmerl, I just can't wait for it. What I'm dealing is not only XML parsing but also Xpath, that's why I can not choose ErlSOM.

The comment on overhead of RPC call's may pay off is exactly right, that's why I only use it for xml parsing and xpath querying. The example for getting time now is just for simple demo.

3. coach factory outlet -- 2012-03-31 06:13

No one can deny the shopping at the coach factory outlet is satisfactory. For the low prices and good quality.Over the years, coach factory online has added a multitude of new handbag shapes, styles and materials to their collection. However, the highest care is taken that every Coach handbags is both aesthetically beautiful and functional.Many fashionable women match with practical Coach Purses which will make the street shopping become relaxed,and make every person can enjoy more diversiform combination in coach factory outlet online.

http://www.ccoachfactoryoutlet.com

4. louis vuitton sale -- 2012-03-31 06:13

Don't feel upset, there will be a great conversion to this kind of situation because there are a large number of louis vuitton sale now!louis vuitton outlet,welcome to buy urban louis vuitton on our online shop.discount price is our special offer, durability and high quality is our promise.louis vuitton Outlets offer famous classic brand for LV,Channel, with perfect service.So become to the VIP soon.They offer more new styles,like LV purses,LV wallets etc. And are tested by product quality monitoring center .

http://www.louisvuittonoutletsaleo.com

5. coach outlet online -- 2012-03-31 06:13

Turn your attention to such discount coach sneakers for women from coach outlet online, you will find something unique and special of such authentic coach for sale at coach factory outlet store online.coach outlet store sells goods that are constructed to meet the highest standards of quality and functionality.You can trust it 100 percent.coach outlet has become a popular shopping experience for consumers around the world, and a desirable distribution channel for manufacturer's and retailers.

http://www.coachoutletonlinesl.com

6. louis vuitton uk -- 2012-03-31 06:39

Our online store offers you discounted Designer louis vuitton replica wallet at present. You could find them in desirable quality and price. If you don't mind high class louis vuitton uk, have a good time here.These is the first time your visit our louis vuit Louis vuitton online shop ,welcome.

http://www.louisvuittonukk.org.uk

7. coach outlet store online -- 2012-03-31 06:41

coach outlet store online marketed properly all greater compared to earth and earn cozy praise from customers. They are made from the finest leather and fabric.coach outlet store have Coach handbags,Coach Shoulder Bags,Coach Briefcases and so on,these bags are so perfectly reproduced,you won't even be able to tell the difference!Coach Outlet Online Store would dynamically change your overall styles right away. The amazing knack about the unique coach handbag is that it would never disappoint your individual styles at all. Rather, it would instantly change your ultimate fashions in a remarkable manner.

http://www.coachoutletstoreonlineeo.com

8. louis vuitton outlet -- 2012-03-31 06:49

any louis vuitton outlet New backpack features usa a function grownup overall look. Varied piece allows that it is captivated me within the approve and also further than your body.It’s induced by way of severe hardworking liver diseases, and also the most familiar factors behind constant louis vuitton bags outlet hard working liver disorder usually are excessive drinking and also liver disease Chemical.Ladies have an ardent love for the Louis Vuitton handbags outlet because Louis Vuitton enjoys a worldwide reputation of high quality and fashionable designs.

http://www.louisvuittonoutletbagsc.com

9. Louis Vuitton Handbags -- 2012-04-15 14:15

Welcome to order Discount Louis Vuitton UK and lead a fashionable, luxury and elegant life from our Louis Vuitton Canada Outlet Store. Enjoy the shopping now here! The Louis Vuitton Handbags are amazing and you will rest assured by proudly owning a bed that you will find yourself envied by simply your colleagues. Louis Vuitton Store Online Handbags can also bring great accuracy as well as practical applicability and fashionable.

http://www.uk-louisvuittonhandbags.co.uk

10. christian louboutin uk -- 2012-04-18 01:32
11. louis vuitton uk -- 2012-04-18 01:42
12. mulberry sale -- 2012-04-18 01:51
13. anonymous -- 2012-08-06 08:18

I am very much happy for Link: online casino slots guide using the great services in this blog and the Link: online gamblingin uk very much happy for the great info Link: online royal casino Thanks a lot for using the amazing services Link: latest poker news in this blog and the great technology.

14. anonymous -- 2012-09-17 06:05

Healing <a href= " http://cheapairjordansa.blogspot.com/ " title= " Cheap Air Jordans " ><strong>Cheap Air Jordans</strong></a>,<a href= " http://retrojordanforcheap.blogspot.com/ " title= " Retro Jordans For Cheap " ><strong>Retro Jordans For Cheap</strong></a>,<a href= " http://cheapjordan11s.blogspot.com/ " title= " Cheap Jordans 11 " ><strong>Cheap Jordans 11</strong></a>. this peony ecstatic. Dark anger Nan Xiao, complex expressions, Chen Sheng said: , a voice echoed this point. voice low and serious, is the man said, let everyone wondering, who in the end come.<a href= " http://womenjordanforcheap.blogspot.com/ " title= " Womens Jordans " ><strong>Womens Jordans</strong></a>,<a href= " http://aorjordansonsale.blogspot.com/ " title= " Jordans on sale " ><strong>Jordans on sale</strong></a> the Dark pondered a moment, Leng Heng said: ; void, the voice said: The figure came from the void in the blink of an eye in front of everyone. ?Should Tianchou opened his eyes, eyes of sorrow looked at exposure to the flames of his brother, sadly sighed: Madness Sangxin tactic is not complete, that is the reason. destruction road , the addition of a struggle of power. time, a voice suddenly in should Tianchou mind in the ring with a fraction indicates the meaning. , only for the commitment these words are said, the day the zombies first saw should Tianchou should Tianchou did not care. Now, recall the words of the serpent god, should Tianchou suddenly realized the original zombies have long seen through his own life also. The Unfortunately own wake up too late, 11 11 a soon as Cui Xiang, dagger fragmentation. a lot of smoke enveloped around, sandwiched bursts of harsh thunderbolt of the thunder in the sky sparks gradually dispersed. When everything is quiet, should Tianchou slowly fall to the ground face not the slightest joy, but some are endless the injury ; sad. ancient saying goes, blood is thicker than water, regret and affection. today, he has personally killed his twin brother, how could not let him feel sad. pay attention to the remnants of snow, should evil slowly squat body, from the snow to pick up a dagger the fragments ; whispered to himself: ; go. &gt; tablets he will never can not forget the sad 11 11 11 ------------ ---- of the century a ------------------ see the sea woman returned, the people around replaced by a leaf favorite, the mirror main Mirage some joy, a warm reception both. Ye favorite, looked at the Mirage, the heart of a strange feeling, as if the person in front of a false and not true. want to spend the wizard from the leaves of your favorite body this feeling, it seems to be sensitive to certain characteristics of the phantom body from {wake of your favorite leaves out mention. The sea woman look innocent, eyes open and innocent, do not understand things. led the duo came to mirror the original boundary, the Phantom asked: The sea woman smiled and said: br&gt; Phantom smiles: do not know, go hurry master. heard of phantom feelings: the battle for thousands of years, finally had a chance to destroy the enemy, of course I can not help but impatient. said: ; Mirage surprised a moment, pondered a moment, smiled and said: to mirror the original community center, where 16 3 mirror a surprising array, the Phantom had in this reception Braun, Aoxue and the sea woman. looked around the leaf favorite: the ; sector. the waved </p>

15. True Religion Jeans Outlet -- 2012-10-11 06:13

Serums and mild hair sprays square measure currently turning into one among the foremost essential beauty merchandise. If you've got long hair, you wish a product that http://www.truereligionjeansoutlet-onsale.com/ True Religion Outlet controls the kink up and keeps long hair stunning and glossy. girls with crisp hair may like a reliable product to take care of the curls and convey out nice volume and bounce. you'll use your Parlux http://clarisonicmia.brushstore0o.com/ Clarisonic Outlet hand blower reception, then bring a hair blood serum or mild spray for a fast fix.

16. anonymous -- 2012-10-12 01:25

Welcome!Coach Outlet Store are loved by many people,when you walk in the street, you could see many people take Coach Shoulder Bags styles.

Coach Factory http://coachfactoryoutlet.1uxury--best.com

Coach Factory Outlet http://www.coachfactoryoutletcd.com

Coach Outlet Store http://www.coachoutletstoreusac.com

Coach Outlet http://www.coachoutletonlinecss.com

17. Canada Goose UK -- 2012-10-12 01:58

The Canada Goose UK are only some of the must have pieces you should definitely include into your accessory parade.

http://www.canada-goose-outlet.co.uk Canada Goose UK

http://www.canadagooseparkas.org Canada Goose Parkas

http://www.canadagoose-sale.org Canada Goose Sale

http://www.canadiagoosejackets.com Canada Goose Jackets

22. anonymous -- 2013-04-28 01:06

​@linlin http://jstlouisvuitton.webs.com/ [Louis Vuitton Outlet] ​http://christianlouboutinblogss.webs.com/ [Christian Louboutin Sale] ​http://jstjordanair.webs.com/ [Authentic Air Jordan Shoes]

24. anonymous -- 2013-07-10 00:59
26. anonymous -- 2013-11-22 08:24

Towering academy and academy students enrolled in http://www.essayscoupon.com/bestdissertation-com-review/ anecdote or craft hikes common own to unscathed intellect plus procreate thesiss. In distribute to mend poke why circumstances happen, understanding and cause piece issues give students an avail to imagine further analytically.

27. anonymous -- 2013-11-23 10:34

It's really a demand to receive any mammoth material processed in alternative tongues, for archetype, C, Java etc. That's cause I faithful to indite RPC server for Erlang, in Java. Design Darlington