Erlang Plugin for NetBeans in Scala#11: Indexer

For IDE support, an indexer is very important for declaration-find, code-completion, re-factory etc. Indexer will monitor the standard lib and your project, analyzing the source/binary files, gather meta-information of classes, functions, global vars, usages, and store them in a search engine (It's a Lucene back-end in NetBeans). That means, you need to support project management and platform management first, so you know where's the class/source path of platform and your current opened projects.

ErlyBird implemented Erlang platform and project management in modules: erlang.platform and erlang.project, these code are derived from Tor's Ruby module, I've migrated them to CSL, but did not rewrote them in Scala. It's OK for integrating these Java written modules with Scala written erlang.editor module.

First, you should identify the class path type, you can register them in ErlangLanguage.scala as:

    /** @see org.netbeans.modules.erlang.platform.ErlangPlatformClassPathProvider and ModuleInstall */
    override
    def getLibraryPathIds = Collections.singleton(BOOT)

    override
    def getSourcePathIds = Collections.singleton(SOURCE)

object ErlangLanguage {
    val BOOT    = "erlang/classpath/boot"
    val COMPILE = "erlang/classpath/compile"
    val EXECUTE = "erlang/classpath/execute"
    val SOURCE  = "erlang/classpath/source"
}

Where, BOOT, thus getLibraryPathIds will point to Erlang OTP libs' path later in:
org.netbeans.modules.erlang.platform.ErlangPlatformClassPathProvider

Now, the indexer itself, ErlangIndex.scala which extends CSL's EmbeddingIndexer and implemented:

override protected def index(indexable:Indexable, parserResult:Result, context:Context) :Unit

with a factory:

class Factory extends EmbeddingIndexerFactory

Register it in ErlangLanguage.scala as:

    override
    def getIndexerFactory = new ErlangIndexer.Factory

Now, tell the CSL to monitor and index them, for Erlang OTP libs, this is done in ModuleInstall.java:

    @Override
    public void restored() {
        GlobalPathRegistry.getDefault().register(ErlangPlatformClassPathProvider.BOOT, new ClassPath[] { ErlangPlatformClassPathProvider.getBootClassPath() });
    }
    
    @Override
    public void uninstalled() {
        GlobalPathRegistry.getDefault().unregister(ErlangPlatformClassPathProvider.BOOT, new ClassPath[] { ErlangPlatformClassPathProvider.getBootClassPath() });
    }

All classpaths registered in GlobalPathRegistry will be monitored and indexed automatically. As "ModuleInstall.java" is also registered in module erlang.platform's manifest.mf as OpenIDE-Module-Install class, it will be automatically loaded and invoked when this module is installed/activated in NetBeans:

Manifest-Version: 1.0
OpenIDE-Module: org.netbeans.modules.erlang.platform
OpenIDE-Module-Layer: org/netbeans/modules/erlang/platform/resources/layer.xml
OpenIDE-Module-Localizing-Bundle: org/netbeans/modules/erlang/platform/Bundle.properties
OpenIDE-Module-Install: org/netbeans/modules/erlang/platform/ModuleInstall.class
OpenIDE-Module-Specification-Version: 0.18.0

For project's classpath, the "GlobalPathRegistry registering is at RubyProject.java (don't be confused by the name, it's a derivation from Ruby's code, I did not change it yet), as:

    private final class ProjectOpenedHookImpl extends ProjectOpenedHook {
        
        ProjectOpenedHookImpl() {}
        
        protected void projectOpened() {
            // register project's classpaths to GlobalPathRegistry
            ClassPathProviderImpl cpProvider = lookup.lookup(ClassPathProviderImpl.class);
            GlobalPathRegistry.getDefault().register(RubyProject.BOOT, cpProvider.getProjectClassPaths(RubyProject.BOOT));
            GlobalPathRegistry.getDefault().register(RubyProject.SOURCE, cpProvider.getProjectClassPaths(RubyProject.SOURCE));            
        }
        
        protected void projectClosed() {
            // unregister project's classpaths to GlobalPathRegistry
            ClassPathProviderImpl cpProvider = lookup.lookup(ClassPathProviderImpl.class);
            GlobalPathRegistry.getDefault().unregister(RubyProject.SOURCE, cpProvider.getProjectClassPaths(RubyProject.SOURCE));
        }
        
    }

So the project's source path will be registered when this project is opened, thus, trigger the index engine.

Now, when you first install Erlang plugin, IDE will index whole OTP libs once. And when an Erlang project is opened, the project's source files will be indexed too.

Then, you need to do a reverse job: search index, get the meta-data to composite Erlang symbols and AST items. This is done in ErlangIndex.scala, which has helper methods to search the index fields, and fetch back the stored signature string, extract it to symbol, and invoke the parsing manager to resolve the AST items. With the help of ErlangIndex.scala, you can search across the whole libs, find completion items, find declarations, find documents embedded in source comments, find usages.

As the first use of index feature, I've implemented the code-completion and go-to-declaration for global modules functions:

When user input "lists:", will invoke code-completion feature, since it's a remote function call, code-completion will search the index data, and find it's from OTP's "lists.erl", after parsing this file, we get all exported functions AstDfn, so, there are a lot of meta-information can be used now, I'll implement the documents tooltips later.

This will be the final one of the series of Erlang plug-in in Scala blogs, thanks for reading and feedback. I'll take a break during the weekend. From the next Monday, with the spring is coming, I'll join a new project, which is an ambitious financial information platform, I'll bring Scala, Lift, Erlang to it. BTW, congratulations to Lift 1.0!