Wednesday Jun 17, 2009

Erlang Plugin Version 1 for NetBeans 6.7 Released

I'm pleased to announce Erlang plugin (ErlyBird) version 1 for NetBeans 6.7 is released.

NetBeans 6.7 RC3 or above is a requirement.

What's new:

  • It's rewritten in Scala instead of Java
  • More reliable instant rename
  • Display extracted document information from source comment when doing auto-completion.

To download, please go to: https://sourceforge.net/project/showfiles.php?group_id=192439&package_id=226387

To install:

  • Open NetBeans, go to "Tools" -> "Plugins", click on "Downloaded" tab title, click on "Add Plugins..." button, choose the directory where the Erlang plugin are unzipped, select all listed *.nbm files, following the instructions.
  • Make sure your Erlang bin path is under OS environment PATH, you can also check/set your OTP path: From [Tools]->[Erlang Platform], fill in the full path of your 'erl.exe' or 'erl' file in "Interpreter", for instance: "C:/erl/bin/erl.exe". Or open the "Brows" dialog to locate the erlang installation.
  • When you open/create an Erlang project first time, the OTP libs will be indexed. Take a coffee and wait, the indexing time varies from 10 to 30 minutes depending on your computer.
Feedback and bug reports are welcome.

Tuesday Jun 02, 2009

Scala Plugin Version 1 for NetBeans 6.7 Released

>>> Updated on July 1, 2009

There are couple of reports on "Could not connect to compilation daemon.", mostly under Windows OS. It's a known issue, to resolve it, please check the following:

  1. SCALA_HOME is set to a fresh installed Scala runtime
  2. PATH includes $SCALA_HOME/bin
  3. If the build task still complain, try to run "fsc" or "scala" in a command window first

===

I'm pleased to announce the availability of Scala plugin version 1 for NetBeans 6.7

What's new:

  • Use fsc instead of scalac as the project building compiler (If you've set SCALA_HOME, make sure $SCALA_HOME/bin is included in your PATH environment).
  • Fixed setting breakpoint in closure statement.
  • A basic import-fixer (Right click on source, then choose "Fix Imports").
  • Code assistant for local vars and functions.
  • Run/Debug single file.

To download, please go to: https://sourceforge.net/project/showfiles.php?group_id=192439&package_id=256544&release_id=686747

For more information, please see http://wiki.netbeans.org/Scala

Bug reports are welcome.

It works on NetBeans 6.7 RC1 or above.

Wednesday May 06, 2009

AIOTrade, Scala for NetBeans, NLP

For past years, I worked on AIOTrade project, which uses Machine Learning technology to find the structural data pattern.

Then, to find a better programming language for AIOTrade, I tried Erlang, Scala, and wrote IDEs for these candidate languages, I learnt a lot of Formal Language Processing.

Now, finally, I dove into Natural Language Processing, and have a good chance to composite the knowledge of Statistics Learning, Rule based Language Processing on massive structural/un-structural data under parallel computing environment.

The candidate programming language is Scala, I keep to improve the NetBeans support for Scala these days while reading the books/code of NLP. During the long weekend, I've got some improvements on it, here is a summary:

  • Migrated to NetBeans' CSL
  • Use fsc instead of scalac as the project building compiler
  • Fixed setting breakpoint in closure statement
  • A basic import-fixer
  • Code assistant for local vars and functions
  • Run/Debug single file

The plugins can be got for beta testing on NetBeans Developing Update Center (for NetBeans 6.7 beta+), and will be released around the date of official NetBeans 6.7.

Sunday Apr 19, 2009

AIOTrade: Progress of Migrating to Scala - DSL and Parallel Computing

Well, I've migrated the core math, indicator parts of AIOTrade to Scala. There are two noticeable advances for new AIOTrade.

The first is the look and feel of indicator writing. Since Scala is suitable for DSL, now the indicator looks like:

class ARBRIndicator extends ContIndicator {
    _sname = "AR/BR"
    _grids = Array(50f, 200f)
    
    val period = Factor("Period", 10)
    
    val up = Var[Float]("up")
    val dn = Var[Float]("dn")
    val bs = Var[Float]("bs")
    val ss = Var[Float]("ss")
    
    val ar = Var[Float]("AR", Plot.Line)
    val br = Var[Float]("BR", Plot.Line)
    
    def computeCont(begIdx:Int, size:Int) {
        for (i <- begIdx until size) {
            up(i) = H(i) - O(i)
            val up_sum_i = sum(i, up, period)
            
            dn(i) = O(i) - L(i)
            val dn_sum_i = sum(i, dn, period)
            
            ar(i) = up_sum_i / dn_sum_i * 100
            
            val bs_tmp = H(i) - C(i)
            bs(i) = Math.max(0, bs_tmp)
            val bs_sum_i = sum(i, bs, period)
            
            val ss_tmp = C(i) - L(i)
            ss(i) = Math.max(0, ss_tmp)
            val ss_sum_i = sum(i, ss, period)
            
            br(i) = bs_sum_i / ss_sum_i * 100
        }
    }   
}  

Vs Java one:

public class ARBRIndicator extends ContIndicator {
    _sname = "AR/BR";
    _grids = new Float[] {50f, 200f};
    
    Opt period = new DefaultOpt("Period", 10);
    
    Var up = new DefaultVar("up");
    Var dn = new DefaultVar("dn");
    Var bs = new DefaultVar("bs");
    Var ss = new DefaultVar("ss");
    
    Var ar = new DefaultVar("AR", Plot.Line);
    Var br = new DefaultVar("BR", Plot.Line);
    
    
    void computeCont(int begIdx) {
        for (int i = begIdx; i < _itemSize; i++) {
            
            up.set(i, H.get(i) - O.get(i));
            float up_sum_i = sum(i, up, period);
            
            dn.set(i, O.get(i) - L.get(i));
            float dn_sum_i = sum(i, dn, period);
            
            ar.set(i, up_sum_i / dn_sum_i * 100);
            
            float bs_tmp = H.get(i) - C.get(i);
            bs.set(i, Math.max(0, bs_tmp));
            float bs_sum_i = sum(i, bs, period);
            
            float ss_tmp = C.get(i) - L.get(i);
            ss.set(i, Math.max(0, ss_tmp));
            float ss_sum_i = sum(i, ss, period);
            
            br.set(i, bs_sum_i / ss_sum_i * 100);
        }
    }
}

The apply method from Scala simplifies setter/getter, which makes the formulator looks more natural.

The second is by implementing each indicator as Actor, the computing procedure of indicators can be easily distributed to multiple CPU cores, with so few code modification:

case object Compute
trait Computable extends Actor {

    // ----- actor's implementation
    def act = loop {
        react {
            case (Compute, fromTime:Long) => computeFrom(fromTime)
            case _ =>
        }
    }
    // ----- end of actor's implementation

    def computeFrom(time:Long) :Unit
    def computedTime :Long
    
    def factors :ArrayBuffer[Factor]
    def factors_=(factors:ArrayBuffer[Factor]) :Unit
    def factors_=(values:Array[Number]) :Unit
    
    def dispose :Unit
}

Computable is an Interface/Trait with sync method: computeFrom(Long), now by extending Computable with Actor, implementing a simple function act with react message processing block, all indicators (which extended Computable) can benefit from parallel computing now by calling:

indicator ! (Compute, fromTime)

instead of

indicator.computeFrom(time)

I've done some testing on my 4-core machine, which occupied about 380% CPU usage during running. This is, of course, a most easily implementation for parallel computing under JVM so far.

Another bonus is, I do not need to worry about concurrent calling on computeFrom(Long) now, since all calls will be triggered by 'Compute' messages that are sent to actor's message queue, then be processed sequentially, there is no lock needed any more. The key:

Parallel computing actors + Sequential message driven computing per actor

Friday Apr 10, 2009

AIOTrade Is Migrating to Scala

Finally, after evaluated Erlang, Scala etc, wrote IDE tools for these languages, I began to migrate AIOTrade from Java to Scala, with the help of Scala for NetBeans of course.

The first step is re-writting basic modules to Scala smoothly, with little functional style code; Then, I'll re-design the APIs by Scala's advanced features, including function, trait, actors etc.

Since AIOTrade is a NetBeans suite project, with several NetBeans style modules integrated, I need a general purpose build.xml to get Scala based modules working. Here are build.xml and scala-build.xml, which can be used to write Scala based NetBeans platform modules.

First, you should create a regular NetBeans module project, then put/replace these ant files under your project's base director. You also need to create 2 NetBeans lib wrapper modules, one is for scala-library.jar, another is for scala-compile.jar as parts of your NetBeans suite project (or, check them out from AIOTrade's source repository)

build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="lib.math" default="netbeans" basedir=".">
    <import file="scala-build.xml"/>
</project>

scala-build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="scala-module" default="netbeans" basedir=".">
    <import file="nbproject/build-impl.xml"/>

    <target name="scala-taskdef" depends="init">
        <property name="scala.library" value="${cluster}/modules/ext/scala-library.jar"/>
        <property name="scala.compiler" value="${cluster}/modules/ext/scala-compiler.jar"/>
        <property name="scala.libs" value="${scala.library}:${scala.compiler}"/>

        <echo message="cluster: ${cluster}"/>
        <echo message="Compiling scala sources via ${scala.library}, ${scala.compiler}"/>

        <taskdef resource="scala/tools/ant/antlib.xml">
            <classpath>
                <pathelement location="${cluster}/modules/ext/scala-library.jar"/>
                <pathelement location="${cluster}/modules/ext/scala-compiler.jar"/>
            </classpath>
        </taskdef>
    </target>

    <property name="jar-excludes" value="**/*.java,**/*.form,**/package.html,**/doc-files/,**/*.scala"/>

    <target name="compile" depends="init,up-to-date,scala-taskdef" unless="is.jar.uptodate">
        <!-- javac's classpath should include scala.library and all these paths of "cp" -->
        <path id="javac.cp">
            <pathelement path="${scala.libs}"/>
            <pathelement path="${module.classpath}"/>
            <pathelement path="${cp.extra}"/>
        </path>
        <!-- scalac will check class dependencies deeply, so we can not rely on public package only which is refed by ${module.classpath} -->
        <path id="scalac.cp">
            <pathelement path="${scala.libs}"/>
            <pathelement path="${module.run.classpath}"/>
            <pathelement path="${cp.extra}"/>
        </path>
        <mkdir dir="${build.classes.dir}"/>
        <depend srcdir="${src.dir}" destdir="${build.classes.dir}" cache="build/depcache">
            <classpath refid="scalac.cp"/>
        </depend>
        <!-- scalac -->
        <scalac srcdir="${src.dir}" destdir="${build.classes.dir}" encoding="UTF-8" target="jvm-${javac.target}">
            <classpath refid="scalac.cp"/>
        </scalac>
        <!-- javac -->
        <nb-javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="${build.compiler.debug}" debuglevel="${build.compiler.debuglevel}" encoding="UTF-8"
                deprecation="${build.compiler.deprecation}" optimize="${build.compiler.optimize}" source="${javac.source}" target="${javac.target}" includeantruntime="false">
            <classpath refid="javac.cp"/>
            <compilerarg line="${javac.compilerargs}"/>
            <processorpath refid="processor.cp"/>
        </nb-javac>
        <!-- Sanity check: -->
        <pathconvert pathsep=":" property="class.files.in.src">
            <path>
                <fileset dir="${src.dir}">
                    <include name="**/*.class"/>
                </fileset>
            </path>
        </pathconvert>
        <fail>
            <condition>
                <not>
                    <equals arg1="${class.files.in.src}" arg2=""/>
                </not>
            </condition>
            You have stray *.class files in ${src.dir} which you must remove.
            Probably you failed to clean your sources before updating them.
        </fail>
        <!-- OK, continue: -->
        <copy todir="${build.classes.dir}">
            <fileset dir="${src.dir}" excludes="${jar-excludes}"/>
        </copy>
    </target>

    <target name="do-test-build" depends="projectized-common.do-test-build">
        <scalac srcdir="${test.unit.src.dir}" destdir="${build.test.unit.classes.dir}" excludes="${test.excludes}"
               encoding="UTF-8">
            <classpath refid="test.unit.cp"/>
        </scalac>
    </target>
</project>

BTW, the new source code of AIOTrade is controlled under Mercurial version control system on sourceforge.net, you can clone or brows the code at: AIOTrade source repository

Note: The whole project can not be successfully built yet.

Friday Mar 27, 2009

AIOTrade Applet Demo Fixed Timezone Issue

I fixed AIOTrade timezone issue. And, the applet demo on aiotrade.com does not need to be a signed applet any more.

For recent experience, I'm re-considering the RIA platform choice. May applet be saved in the near future? or Flash, Javascript win? First, Java applet needs a whole new look and feel theme, which make it fitting into the web page, second, how can a Java applet access DOM tree easily?

But,

When Flash/Flex tries to take the place of Java applet, it will become another Java itself.

Saturday Mar 14, 2009

New Face of aiotrade.com with AIOTrade Applet Demo

I've done some preliminary re-design on aiotrade.com, put the AIOTrade applet on home page, it requires browser with Java plugin above JRE 1.5, the size is about 288k. You can input the symbol to get real-time quote charting. Symbol is same as Yahoo! finance. The quote data is also from Yahoo! finance.

There are still some bugs on it. I'll continue to improve it.

Here's the snapshot of whole home page after you input a symbol. You can remove the chart by press "Remove Quotes" button.

nn

Wednesday Mar 11, 2009

AIOTrade - It's not Flash or A Picture It's A Java Applet on Web Page

For RIA framework, there are choices of Flex/Flash, Applet/JavaFX etc. I saw a lot of real-time financial charts were written in Flex/Flash, all over the world wide web. Then, how about a Java applet financial charting after Java 6u10? I'd like a try, and here is the result:

  • The whole size of a most featured AIOTrade is less than 300k
  • On my Macbook, with Java 6u12 installed, the init time is about 1 sec.
  • By properly arranging Swing layout, the applet can auto-scale to proper size fit in html page just like any element.
Below is a snapshot:

nn

Friday Feb 27, 2009

Erlang Plugin for NetBeans in Scala#11: Indexer

For IDE support, an indexer is very important for declaration-find, code-completion, re-factory etc. Indexer will monitor the standard lib and your project, analyzing the source/binary files, gather meta-information of classes, functions, global vars, usages, and store them in a search engine (It's a Lucene back-end in NetBeans). That means, you need to support project management and platform management first, so you know where's the class/source path of platform and your current opened projects.

ErlyBird implemented Erlang platform and project management in modules: erlang.platform and erlang.project, these code are derived from Tor's Ruby module, I've migrated them to CSL, but did not rewrote them in Scala. It's OK for integrating these Java written modules with Scala written erlang.editor module.

First, you should identify the class path type, you can register them in ErlangLanguage.scala as:

    /** @see org.netbeans.modules.erlang.platform.ErlangPlatformClassPathProvider and ModuleInstall */
    override
    def getLibraryPathIds = Collections.singleton(BOOT)

    override
    def getSourcePathIds = Collections.singleton(SOURCE)

object ErlangLanguage {
    val BOOT    = "erlang/classpath/boot"
    val COMPILE = "erlang/classpath/compile"
    val EXECUTE = "erlang/classpath/execute"
    val SOURCE  = "erlang/classpath/source"
}

Where, BOOT, thus getLibraryPathIds will point to Erlang OTP libs' path later in:
org.netbeans.modules.erlang.platform.ErlangPlatformClassPathProvider

Now, the indexer itself, ErlangIndex.scala which extends CSL's EmbeddingIndexer and implemented:

override protected def index(indexable:Indexable, parserResult:Result, context:Context) :Unit

with a factory:

class Factory extends EmbeddingIndexerFactory

Register it in ErlangLanguage.scala as:

    override
    def getIndexerFactory = new ErlangIndexer.Factory

Now, tell the CSL to monitor and index them, for Erlang OTP libs, this is done in ModuleInstall.java:

    @Override
    public void restored() {
        GlobalPathRegistry.getDefault().register(ErlangPlatformClassPathProvider.BOOT, new ClassPath[] { ErlangPlatformClassPathProvider.getBootClassPath() });
    }
    
    @Override
    public void uninstalled() {
        GlobalPathRegistry.getDefault().unregister(ErlangPlatformClassPathProvider.BOOT, new ClassPath[] { ErlangPlatformClassPathProvider.getBootClassPath() });
    }

All classpaths registered in GlobalPathRegistry will be monitored and indexed automatically. As "ModuleInstall.java" is also registered in module erlang.platform's manifest.mf as OpenIDE-Module-Install class, it will be automatically loaded and invoked when this module is installed/activated in NetBeans:

Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.erlang.platform
OpenIDE-Module-Layer: org/netbeans/modules/erlang/platform/resources/layer.xml
OpenIDE-Module-Localizing-Bundle: org/netbeans/modules/erlang/platform/Bundle.properties
OpenIDE-Module-Install: org/netbeans/modules/erlang/platform/ModuleInstall.class
OpenIDE-Module-Specification-Version: 0.18.0

For project's classpath, the "GlobalPathRegistry registering is at RubyProject.java (don't be confused by the name, it's a derivation from Ruby's code, I did not change it yet), as:

    private final class ProjectOpenedHookImpl extends ProjectOpenedHook {
        
        ProjectOpenedHookImpl() {}
        
        protected void projectOpened() {
            // register project's classpaths to GlobalPathRegistry
            ClassPathProviderImpl cpProvider = lookup.lookup(ClassPathProviderImpl.class);
            GlobalPathRegistry.getDefault().register(RubyProject.BOOT, cpProvider.getProjectClassPaths(RubyProject.BOOT));
            GlobalPathRegistry.getDefault().register(RubyProject.SOURCE, cpProvider.getProjectClassPaths(RubyProject.SOURCE));            
        }
        
        protected void projectClosed() {
            // unregister project's classpaths to GlobalPathRegistry
            ClassPathProviderImpl cpProvider = lookup.lookup(ClassPathProviderImpl.class);
            GlobalPathRegistry.getDefault().unregister(RubyProject.SOURCE, cpProvider.getProjectClassPaths(RubyProject.SOURCE));
        }
        
    }

So the project's source path will be registered when this project is opened, thus, trigger the index engine.

Now, when you first install Erlang plugin, IDE will index whole OTP libs once. And when an Erlang project is opened, the project's source files will be indexed too.

Then, you need to do a reverse job: search index, get the meta-data to composite Erlang symbols and AST items. This is done in ErlangIndex.scala, which has helper methods to search the index fields, and fetch back the stored signature string, extract it to symbol, and invoke the parsing manager to resolve the AST items. With the help of ErlangIndex.scala, you can search across the whole libs, find completion items, find declarations, find documents embedded in source comments, find usages.

As the first use of index feature, I've implemented the code-completion and go-to-declaration for global modules functions:

nn

When user input "lists:", will invoke code-completion feature, since it's a remote function call, code-completion will search the index data, and find it's from OTP's "lists.erl", after parsing this file, we get all exported functions AstDfn, so, there are a lot of meta-information can be used now, I'll implement the documents tooltips later.

This will be the final one of the series of Erlang plug-in in Scala blogs, thanks for reading and feedback. I'll take a break during the weekend. From the next Monday, with the spring is coming, I'll join a new project, which is an ambitious financial information platform, I'll bring Scala, Lift, Erlang to it. BTW, congratulations to Lift 1.0!

Erlang Plugin for NetBeans in Scala#10: Code Completion

Implementing Code-Completion is a bit complex, but you can got it work gradually. At the first step, you can implement Code-Completion for local vars/functions only, then, with the indexed supporting, you can add completion for remote functions.

You should define some kinds of completion proposal, which may show different behaviors when they are popped up and guard you followed steps. For example, a function proposal can auto-fill parameters, on the other side, a keyword proposal just complete itself.

The completion proposal classes are defined in ErlangComplectionProposal.scala, which implemented CSL's interface CompletionProposal. you may notice that the function proposal is the most complex one, which should handle parameters information.

Then, you should implement CSL's interface CodeCompletionHandler, for Erlang, it's ErlangCodeCompletion, where, the key method is:

    override
    def complete(context:CodeCompletionContext) :CodeCompletionResult = {
        this.caseSensitive = context.isCaseSensitive
        val pResult = context.getParserResult.asInstanceOf[ErlangParserResult]
        val lexOffset = context.getCaretOffset
        val prefix = context.getPrefix match {
            case null => ""
            case x => x
        }

        val kind = if (context.isPrefixMatch) QuerySupport.Kind.PREFIX else QuerySupport.Kind.EXACT
        val queryType = context.getQueryType

        val doc = LexUtil.document(pResult, true) match {
            case None => return CodeCompletionResult.NONE
            case Some(x) => x.asInstanceOf[BaseDocument]
        }

        val proposals = new ArrayList[CompletionProposal]
        val completionResult = new DefaultCompletionResult(proposals, false)

        // Read-lock due to Token hierarchy use
        doc.readLock
        try {
            val astOffset = LexUtil.astOffset(pResult, lexOffset)
            if (astOffset == -1) {
                return CodeCompletionResult.NONE
            }
            val root = pResult.rootScope match {
                case None => return CodeCompletionResult.NONE
                case Some(x) => x
            }
            val th = LexUtil.tokenHierarchy(pResult).get
            val fileObject = LexUtil.fileObject(pResult).get

            val request = new CompletionRequest
            request.completionResult = completionResult
            request.result = pResult
            request.lexOffset = lexOffset
            request.astOffset = astOffset
            request.index = ErlangIndex.get(pResult)
            request.doc = doc
            request.info = pResult
            request.prefix = prefix
            request.th = th
            request.kind = kind
            request.queryType = queryType
            request.fileObject = fileObject
            request.anchor = lexOffset - prefix.length
            request.root = root
            ErlangCodeCompletion.request = request
            
            val token = LexUtil.token(doc, lexOffset - 1) match {
                case None => return completionResult
                case Some(x) => x
            }

            token.id match {
                case ErlangTokenId.LineComment =>
                    // TODO - Complete symbols in comments?
                    return completionResult
                case ErlangTokenId.StringLiteral =>
                    //completeStrings(proposals, request)
                    return completionResult
                case _ =>
            }
            
            val ts = LexUtil.tokenSequence(th, lexOffset - 1) match {
                case None => return completionResult
                case Some(x) =>
                    x.move(lexOffset - 1)
                    if (!x.moveNext && !x.movePrevious) {
                        return completionResult
                    }
                    x
            }
 
            val closetToken = LexUtil.findPreviousNonWsNonComment(ts)

            if (root != null) {
                val sanitizedRange = pResult.sanitizedRange
                val offset = if (sanitizedRange != OffsetRange.NONE && sanitizedRange.containsInclusive(astOffset)) {
                    sanitizedRange.getStart
                } else astOffset

                val call = Call(null, null, false)
                findCall(root, ts, th, call, 0)
                val prefixBak = request.prefix
                call match {
                    case Call(null, _, _) =>
                    case Call(base, _, false) =>
                        // it's not a call, but may be candicate for module name, try to get modules and go-on
                        completeModules(base, proposals, request)
                    case Call(base, select, true) =>
                        if (select != null) {
                            request.prefix = call.select.text.toString
                        } else {
                            request.prefix = ""
                        }
                        completeModuleFunctions(call.base, proposals, request)
                        // Since is after a ":", we won't added other proposals, just return now whatever
                        return completionResult
                }
                request.prefix = prefixBak
                completeLocals(proposals, request)
            }

            completeKeywords(proposals, request)
        } finally {
            doc.readUnlock
        }

        completionResult
    }

For a Erlang function call, you should check the tokens surrounding the caret to get the call's base name and select first, which is done by a method findCall:

    private def findCall(rootScope:AstRootScope, ts:TokenSequence[TokenId], th:TokenHierarchy[_], call:Call, times:Int) :Unit = {
        assert(rootScope != null)
        val closest = LexUtil.findPreviousNonWsNonComment(ts)
        val idToken = closest.id match {
            case ErlangTokenId.Colon =>
                call.caretAfterColon = true
                // skip RParen if it's the previous
                if (ts.movePrevious) {
                    val prev = LexUtil.findPreviousNonWs(ts)
                    if (prev != null) {
                        prev.id match {
                            case ErlangTokenId.RParen   => LexUtil.skipPair(ts, ErlangTokenId.LParen,   ErlangTokenId.RParen,   true)
                            case ErlangTokenId.RBrace   => LexUtil.skipPair(ts, ErlangTokenId.LBrace,   ErlangTokenId.RBrace,   true)
                            case ErlangTokenId.RBracket => LexUtil.skipPair(ts, ErlangTokenId.LBracket, ErlangTokenId.RBracket, true)
                            case _ =>
                        }
                    }
                }
                LexUtil.findPrevIncluding(ts, LexUtil.CALL_IDs)
            case id if LexUtil.CALL_IDs.contains(id) => closest
            case _ => null
        }

        if (idToken != null) {
            times match {
                case 0 if call.caretAfterColon => call.base = idToken
                case 0 if ts.movePrevious => LexUtil.findPreviousNonWsNonComment(ts) match {
                        case null => call.base = idToken
                        case prev if prev.id == ErlangTokenId.Colon =>
                            call.caretAfterColon = true
                            call.select = idToken
                            findCall(rootScope, ts, th, call, times + 1)
                        case _ => call.base = idToken
                    }
                case _ => call.base = idToken
            }
        }
    }

    case class Call(var base:Token[TokenId], var select:Token[TokenId], var caretAfterColon:Boolean)

To complete a remote function call, you may need to visit outer modules, which needs an indexer, so as the first step, you can just ignore it, go straight to complete local vars/functions, or keywords:

    private def completeLocals(proposals:List[CompletionProposal], request:CompletionRequest) :Unit = {
        val prefix = request.prefix
        val kind = request.kind
        val pResult = request.result

        val root = request.root
        val closestScope = root.closestScope(request.th, request.astOffset) match {
            case None => return
            case Some(x) => x
        }
        val localVars = closestScope.visibleDfns(ElementKind.VARIABLE)
        localVars ++= closestScope.visibleDfns(ElementKind.PARAMETER)
        localVars.filter{v => filterKind(kind, prefix, v.name)}.foreach{v =>
            proposals.add(new PlainProposal(v, request.anchor))
        }

        val localFuns = closestScope.visibleDfns(ElementKind.METHOD)
        localFuns.filter{f => filterKind(kind, prefix, f.name)}.foreach{f =>
            proposals.add(new FunctionProposal(f, request.anchor))
        }
    }

    private def completeKeywords(proposals:List[CompletionProposal], request:CompletionRequest) :Unit = {
        val prefix = request.prefix
        val itr = LexerErlang.ERLANG_KEYWORDS.iterator
        while (itr.hasNext) {
            val keyword = itr.next
            if (startsWith(keyword, prefix)) {
                proposals.add(new KeywordProposal(keyword, null, request.anchor))
            }
        }
    }

There is a function "def visibleDfns(kind:ElementKind) :ArrayBuffer[AstDfn]" in AstRootScope.scala, if you've put definition items properly in scopes, it should handle the visibility automatically.

Now, register it in ErlangLanguage.scala:

    override
    def getCompletionHandler = new ErlangCodeCompletion

As usual, run it, you got:

nn

The local functions and vars are proposed plus the keywords. BTW, I've fixed this feature for Scala plug-in.

Scala Corner Case#3: "object" or "case object" Extends Case Class? It's a Different Story

Case class in Scala is a neat feature, it automatically generates apply, extract and equals functions. But I encountered a strange "equals" behavior today, which wasted me 2 hours to find why.

Define a simple case class "Symbol", then a "VarSymbol" which extends "Symbol" with one more field "name:String". We know you can compare the equality of instances of "VarSymbol" by this "name", Scala automatically gets it right. And of course, a direct instance from "Symbol()" should not equal to any "VarSymbol", since it lacks "name" field, that's right.

case class Symbol()
case class VarSymbol(name:String) extends Symbol

Then, I need a singleton object "NoSymbol", I defined it as:

object NoSymbol extends Symbol

I of course thought this "NoSymbol" should not equal any "VarSymbol" instance, or,

NoSymbol == VarSymbol("I'm var")

Should return false. But life is not so straightforward, it returns true in real life !!!

I finally got my code working, by adding a "case" before "object NoSymbol extends Symbol". This "NoSymbol" won't equal any "VarSymbol" now, I'm grad things finally go back in track.

I don't know if it's a Scala bug, or, that's what Scala thinks it should be: all "Symbol" and it's inherited instances should equal "NoSymbol" object. Anyway, here's the whole code for test:

case class Symbol()
case class VarSymbol(name:String) extends Symbol

object NoSymbol extends Symbol
case object CasedNoSymbol extends Symbol

object TestCaseObject {
    
    def test = {
        val noSym = NoSymbol
        val caseNoSym = CasedNoSymbol
        val varSym = VarSymbol("I'm var")

        if (noSym == varSym) println("NoSym equals varSym !") 
        else println("NoSym doesn't equal varSym")
        
        if (caseNoSym == varSym) println("CaseNoSym equals varSym !") 
        else println("CaseNoSym doesn't equal varSym")
    }
    
}

Run TestCaseObject.test, I got:

NoSym equals varSym !
CaseNoSym doesn't equal varSym

Monday Feb 23, 2009

Better Look&Feel of NetBeans on Mac OS

NetBeans 6.7M2 is going to be public available soon, the new look&feel on Mac OS is very awesome. BTW, "-Dapple.awt.graphics.UseQuartz=true" is a must option for best font-rendering on my macbook. I added it to netbeans.conf as "-J-Dapple.awt.graphics.UseQuartz=true". Here is a snapshot of my current screen when working on Erlang plugin in Scala:

Click on the picture to enlarge it:

nn

Saturday Feb 21, 2009

Scala Corner Case#2: "Nothing" Can not Be Cast and Assigned to a val/var

There is a Java class which takes type parameter T, T is any type:

package dcaoyuan.test;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

public class AList<T> implements Iterator<T> {

   List<T> a = new ArrayList<T>();

   {
       a.add((T) "string");
   }

   Iterator<T> itr = a.iterator();

   public boolean hasNext() {
       return itr.hasNext();
   }

   public T next() {
       return itr.next();
   }

   public void remove() {
       throw new UnsupportedOperationException("Not supported yet.");
   }
}

Which implemented java.util.Iterator with type parametered T next(). Notice here next() will return T

The above code simulates a case, that an existed Java lib may return an instance that has type parameter erased, on run-time.

Now let's try to call it in Scala:

package dcaoyuan.test

object Main {

   def main(args: Array[String]) :Unit = {
       testAList_work
       testAList_unwork
   }

   def testAList_work :Unit = {
       val a :AList[_] = new AList
       // a is inferred as AList[Any]
       while (a.hasNext) {
           val e = a.next
           println(e)
       }
   }

   def testAList_unwork :Unit = {
       val a = new AList
       // a is inferred as AList[Nothing]
       while (a.hasNext) {
           // now a.next is Nothing, can not cast to "val e", which should at least be Any
           val e = a.next // will throw java.lang.ClassCastException
           println(e)
       }
   }
}

I wrote two functions: "testAList_work" and "testAList_unwork"

The issue is in testAList_unwork, as a simple "val a = new AList", "a" will be inferred by scalac as AList[Nothing], so, "a.next" will return an instance of type Nothing, then, "val e = a.next" will throw java.lang.ClassCastException

My question is, should this corner case be checked by scalac as an error or warning? So I do not need to dig the clause when it happens on run-time.

Friday Feb 20, 2009

Scala Corner Case#1: Implement Method of Java Interface with Type Parameter Omitted by It's Sub-Class

The progress of rewriting Erlang plugin for NetBeans in Scala has reached a phase, that the Editor itself works smooth and better than ErlyBird now, the next step is to integrate an Erlang project management and index the modules/functions of OTP/project to provide smart auto-completion.

Now Scala has been proved that it can be integrated into an existed large Java based framework (NetBeans IDE here) without no much problems, I'll begin to rewrite Scala plugin in Scala soon.

Although in most cases, Scala can call existed Java code or classes smoothly, there are still some corner cases. I'll record these corner cases in blogs. Here's the first one which I also posted on scala-user mailing-list, but have not yet got final answer.

Let's begin with a detailed example:

There is a Java interface A:

public interface A<T extends String> {
   void run(T t);
}

Which has a type parameter <T extends String> and abstract method run(T t)

Then a Java abstract class B extended A. But, B, as it, omitted type parameter from A. This is unsafe but valid in Java:

public abstract class B implements A {
   public String me() {
       return "I'm B";
   }
}

Assume above classes A and B have been compiled under javac, and packed in a jar library, and I can not patch it anymore. Now I need to write a class S in Scala which should extend B:

class S extends B {
   override
   def run[T <: String](t:T) = {println(t)}
}

scalac will complain as:

/Users/dcaoyuan/NetBeansProjects/ScalaTestCase/src/S.scala:1: error:
class S needs to be abstract, since method run in trait A of type
(T)Unit is not defined
class S extends B {
/Users/dcaoyuan/NetBeansProjects/ScalaTestCase/src/S.scala:3: error:
method run overrides nothing
   def run[T <: String](t:T) = {println(t)}

I than tried "forSome" type:

class S extends B {
   override
   def run(t:T forSome {type T <: String}) = {println(t)}
}

The code still did not work.

It seems that, since B omitted A's type parameter T, I have no way to get what is T, and can not successfully implement "run(T t)" method.

I also tried other forms of "forSome" usages, and always failed.

But I think Scala can always be saved with mixed Java/Scala code in such corner case, that's what I believed. So, I thought about it later when my brain was spare, and finally got a solution:

I wrote another Java abstract class B1 which extends B and pretended to have implemented "run(T t)", but actually called another new abstract method "runImpl(String t)"

public abstract class B1 extends B {

    public void run(String t) {
        runImpl(t);
    }

    public abstract void runImpl(String t);
}

Now I can let Scala S extends B1 and implement "runImpl(String t)" instead of extending B and implementing "run(T t)".

class S extends B1 {
    override
    def runImpl(t:String) = {println(t)}
}

Yes, scalac won't complain about "runImpl(t:String)" at all, and I got S successfully extends B by bridge class B1.

But I still hope scalac can resolve it directly, with a warning message instead of failing to compile it.