Caoyuan's Blog

Parsing Performance of Scala for NetBeans

I'm re-considering the indexing mechanism of Scala for NetBeans. The indexing is used to store meta-info of parsed templates/methods etc, for auto-completion and document/source offsets searching. Currently, the parsing phases include lexer, syntax and semantic analysis, not include type inference and type check.

With a basic performance testing on all Scala standard library and liftweb's library, the maxima parsing time seems less than 1s, the average parsing time is around 0.2s, not bad.

So, this may let the indexing feature a lot simple, I can store classes/objects/traits' meta-info only, instead of including their type parameters and their members (fields/methods, scoped importing etc), these additional information can be got via re-parsing the source file or querying the class file.

Posted at 08:30PM Jun 10, 2008 by dcaoyuan in NetBeans  | 

Bundled Latest Scala Runtime to Scala for NetBeans

I just bundled latest Scala runtime to Scala for NetBeans, the version is 2.7.1.final, this brings two things:

First, you do not need to set SCALA_HOME any more to get whole plugins working. But, you can still set SCALA_HOME to specifying the target Scala version, if so, you should also need to download and unzip source jars to $SCALA_HOME/src;

Second, I'll begin to write some code in Scala instead of Java for Scala plugins. I can evaluate the features of Scala plugins in daily work, find and fix more bugs of plugins.

Posted at 08:54AM May 27, 2008 by dcaoyuan in NetBeans  |  Comments[9]

Scala for NetBeans Screenshot#12: Better Completion with More Types Inferred

>>> Updated May 11
Mostly, infix expressions can be type inferred now. CTRL+Click on infix op name, it will bring you to declaration (since 1.0.29.0)
===

Well, the type inference work is not so easy (with performance in mind), but anyway, I've got a bit more progress, at least, the chained member call can now be correctly inferred in a lot of cases. It's some level as Tor's JavaScript for NetBeans now.

First, let's create a val "node", which is a "scala.xml.Node"

Then, input '.' to invoke completion, as I know which type is of "node", the proposal items look good.

I choose "descendant" function (which returns a "List"), and input '.' again, we can see the proposal items look still good.

These features also work on Java's class.

Known issues:

• It seems the indexing/scanning for Scala standard library source will perform twice when you first installed Scala plugins
• The type inference is not consistence yet, so don't be strange for the strange behavior sometimes

Again, don't forget to download scala standard library's source jars and unzip to $SCALA_HOME/src, per sub-folder per jar

Posted at 10:45AM May 09, 2008 by dcaoyuan in NetBeans  |  Comments[6]

Which Programming Language J. Gosling Would Use Now, Except Java?

>>> Updated later
Maybe we can get completeness of J. Gosling's opinions about Java/Scaka/JVM from here
===

According to Adam Bien's blog from JavaOne

During a meeting in the Community Corner (java.net booth) with James Gosling, a participant asked an interesting question: "Which Programming Language would you use *now* on top of JVM, except Java?". The answer was surprisingly fast and very clear: - Scala.

I think Fortress will also be a very good future choice when it gets mature.

Posted at 07:39AM May 08, 2008 by dcaoyuan in Java  | 

Scala for NetBeans Screenshot#11: Go to Remote Declaration and Doc Tooltip

Two crazy days of my spare time, I was coding between sleeping and eating, with a lot of cups of coffee. Now Scala plugins support Go-To remote declarations (CTRL + Click on var/function name), and when you put cursor on the identifier name with CTRL pressed, or under auto-completion, the doc will also be shown as tooltip.

These features work for Java classes too with a bit poor performance, I'll fix it later (fixed).

Not all identifiers have been type inferred, so these features are not applicable for all identifiers.

Please update to Scala Editing module as version 1.0.26.xxx when it's available, which is the only stable one these days. Remember to unzip Scala lib's source under $SCALA_HOME/src

Posted at 12:04AM May 03, 2008 by dcaoyuan in NetBeans  |  Comments[8]

Scala for NetBeans Screenshot#10: Working on Auto-Completion for Java

I've done some hacking work to get Java classes to be completion enabled, but it's not full functional. Any way, it's a good start point for auto-completion for Scala plugins, I hope to get more type inference work to be finished, and finally support both Scala/Java classes smart-completion.

Click on the picture to enlarge it

Posted at 08:09PM Apr 29, 2008 by dcaoyuan in NetBeans  |  Comments[3]

Scala for NetBeans Screenshot#9: Working on Auto-Completion

With the indexed cache of project's Class/Trait/Object, and Scala standard library's source files, the auto-completion is a bit smarter now. If the val/var is defined with type, the auto-completion can know which methods will be suggested. (Not work for java classes yet)

To get this working, you should follow these steps:

• Update to newest Scala plugins (Editing version 1.0.21.1)
• Delete the old-cache files which are located at your NetBeans's configuration directory (for example, .netbeans/dev/var/cache).
• Download Scala standard library's source file, unzip them to $SCALA_HOME/src, per sub-folder per source jar file

Click on the picture to enlarge it

Posted at 07:40PM Apr 25, 2008 by dcaoyuan in NetBeans  |  Comments[4]

New Scala Plugins for NetBeans are Available for Public Test, and Fortress, Erlang

>>> Updated Apr 22
Due to the incompatibly changes of NetBeans underlaying modules, Scala plugins can not be installed/updated on NetBeans 6.1 any more, you should get the latest nightly build to play with Scala plugins.
===

>>> Updated Apr 19

• Fixed some broken syntax
• Fixed NPE caused by brace completion
• Fixed indentation of case class/object
• Added formatting options ("Preference" -> "Scala" -> "Formatting")

The new written Scala plugins for NetBeans are available for public test now, which can be installed on NetBeans 6.1 RC, and latest NetBeans nightly build. To get start, please visit http://wiki.netbeans.org/Scala

The following features are ready for test:

• Syntax highlighting
• Auto-indentation
• Brace completion
• Formatter
• Outline navigator
• Occurrences mark for local variables and functions
• Instance rename for local variables and functions
• Go-to-declaration for local variables and functions
• Scala project
• Basic debugger

And with known issues:

• Auto-completion it not fully supported yet and not smart
• There is no parsing errors recovering yet
• Semantic errors are not checked on editing, but will be noticed when you build project
• Due to the un-consistent of Scala's grammar reference document, there may be some syntax broken issues

BTW, Fortress editing plugin is also available on "Last Development Build" update center, see the installation part of http://wiki.netbeans.org/Scala to get it installed. It's a very alpha stage plugin.

And, Erlang plugins are also available from "Last Development Build" update center too, that is, you can install and use Erlang plugins with Ruby, Scala, JavaScript on the same NetBeans IDE (6.1 RC or nightly build). Thanks to Tor's work, the indexing performance has been improved a lot.

Erlang plugin will be rewritten in the near future too.

Posted at 09:21AM Apr 18, 2008 by dcaoyuan in NetBeans  |  Comments[13]

Scala for NetBeans Screenshot#8: Working on Indexer

I've done some basic indexer code, that is, all source files under a project will be parsed, analyzed, then indexed (class/object/trait, functions, fields etc). But it's just a start, before I finished type inference, if you press CTRL+SPACE to invoke completion, there are a lot of indexed Class/Object/Trait/Function will be roughly shown on you :-), it's not smart, it's more like a puzzle, you should decide which one is applicable by yourself. But you can get a view of the coming completion feature.

Click on the picture to enlarge it

Posted at 03:08PM Apr 14, 2008 by dcaoyuan in NetBeans  |  Comments[4]

Progress of Scala for NetBeans - with a New Written Lexer and Parser

According to this post:

I talked to Jetbrains about this, and they told me that they stopped working on the Scala plugin for the time being, because - demand for Groovy/Ruby was higher - the language was moving too fast - Scala is a terribly difficult language for compiler/tool writers, and the only good way to analyze Scala programs might be through the official compiler, which didn't yet support this

It's true that "Scala is a terribly difficult language for compiler/tool writers", but I'm trying to bypass "the only good way to analyze Scala programs might be through the official compiler"

Before rewriting Scala for NetBeans, I considered some parser choices, one was Scala's native compiler, which is good for compiling/building Scala project, but not suitable for Editor. And JavaCC, ANTLR, which may be good enough, but it's not natural to express Scala's grammar.

Then I found Rats! which is used by Fortress, a very very clean, powerful parser generator. After couple of days working, I got an incremental lexer for Scala, and a parser for Scala that with Scala's grammar being naturally expressed (the grammar definition is ParserScala.rats). The benefit of a complete controllable parser is that I can now do some type inference and wholly semantic analysis freely and immediately.

Another progress is that I've decoupled the Scala project's dependency on Java.source's classpath in NetBeans, instead, GSF's classpath is used in Scala project module now. That means, I can begin the indexer for Scala's standard library and project source files.

The next steps will be type inference; smart completion with type inferred information; indexer for later refectory and usages searching; parsing error recover etc.

Posted at 07:20AM Apr 13, 2008 by dcaoyuan in NetBeans  |  Comments[4]

Where We Are - Rewriting Scala for NetBeans

>>> Updated:
The rewritten plugin will be available, depending on the NetBeans' nightly build, it may be available after 10 hours or more. New features include a formatter (CTRL+SHIFT+F), and better brace completer. Smart auto-completion does not work now, needs further working.
===

The new Scala for NetBeans is going to form a good shape, I've got syntax highlighting, indentation, formatting, brace matching, and basic structure outline working. Here's a snapshot showing complex Scala statements being highlighted and reformatted properly:

Posted at 01:58PM Mar 31, 2008 by dcaoyuan in NetBeans  |  Comments[1]

Begin Rewriting Scala for NetBeans

I'm re-writing Scala for NetBeans. Everything is broken except syntax highlighting.

Please be patient to wait for further progress.

Posted at 10:30PM Mar 23, 2008 by dcaoyuan in NetBeans  |  Comments[2]

Developing IDE Based on GSF for NetBeans#1 - Minimal Support

There has been GSF (Generic Scripting Framework) which is Tor's working derived and abstracted from Java supporting code, and, the base of Ruby/JavaScript support for NetBeans.

So, how to develop an IDE based on GSF for NetBeans? I'd like to share some experiences in this series of articles, a series of outline description, without too much code and details, for detailed information, please go into the source code on hg.netbeans.org

I. Minimal Support - Highlighting

1. public class ScalaLanguage implements GsfLanguage
2. public class ScalaMimeResolver extends MIMEResolver
3. public enum ScalaTokenId implements TokenId
4. public class ScalaLexer implements Lexer<ScalaTokenId>

Then, register your language in layer.xml:

<filesystem>
    <folder name="Editors">
        <folder name="text">
            <folder name="x-scala">
                <attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>

                <file name="language.instance">
                    <attr name="instanceCreate" methodvalue="org.netbeans.modules.scala.editing.lexer.ScalaTokenId.language"/>
                    <attr name="instanceOf" stringvalue="org.netbeans.api.lexer.Language"/>
                </file>
                
                <folder name="FontsColors">
                    <folder name="NetBeans">
                        <folder name="Defaults">
                            <file name="coloring.xml" url="fontsColors.xml">
                                <attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>
                            </file>
                        </folder>
                    </folder>
                </folder>
                
                <folder name="CodeTemplates">
                    <folder name="Defaults">
                        <file name="codetemplates.xml" url="codetemplates.xml">
                            <attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>
                        </file>
                    </folder>
                </folder>

                <folder name="Keybindings">
                    <folder name="NetBeans">
                        <folder name="Defaults">
                            <file name="org-netbeans-modules-scala-editing-keybindings.xml" url="DefaultKeyBindings.xml"/>
                        </folder>
                    </folder>
                </folder>
            </folder>
        </folder>
    </folder>
      
    <folder name="GsfPlugins">
        <folder name="text">
            <folder name="x-scala">
                <file name="language.instance">
                    <attr name="instanceOf" stringvalue="org.netbeans.modules.gsf.api.GsfLanguage"/>
                    <attr name="instanceClass" stringvalue="org.netbeans.modules.scala.editing.ScalaLanguage"/>
                </file>
            </folder>
        </folder>
    </folder>
    
    <folder name="Loaders">
        <folder name="text">
            <folder name="x-scala">
                <attr name="SystemFileSystem.icon" urlvalue="nbresloc:/org/netbeans/modules/scala/editing/resources/scala16x16.png"/>
                <attr name="iconBase" stringvalue="org/netbeans/modules/scala/editing/resources/scala16x16.png"/>
                <folder name="Actions">
                    <file name="OpenAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.OpenAction"/>
                        <attr name="position" intvalue="100"/>
                    </file>
                    <file name="Separator1.instance">
                        <attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
                        <attr name="position" intvalue="200"/>
                    </file>
                    <file name="CutAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.CutAction"/>
                        <attr name="position" intvalue="300"/>
                    </file>     
                    <file name="CopyAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.CopyAction"/>
                        <attr name="position" intvalue="400"/>
                    </file>
                    <file name="PasteAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.PasteAction"/>
                        <attr name="position" intvalue="500"/>
                    </file>
                    <file name="Separator2.instance">
                        <attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
                        <attr name="position" intvalue="600"/>
                    </file>
                    <file name="NewAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.NewAction"/>
                        <attr name="position" intvalue="700"/>
                    </file>
                    <file name="DeleteAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.DeleteAction"/>
                        <attr name="position" intvalue="800"/>
                    </file>
                    <file name="RenameAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.RenameAction"/>
                        <attr name="position" intvalue="900"/>
                    </file>
                    <file name="Separator3.instance">
                        <attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
                        <attr name="position" intvalue="1000"/>
                    </file>
                    <file name="SaveAsTemplateAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.SaveAsTemplateAction"/>
                        <attr name="position" intvalue="1100"/>
                    </file>
                    <file name="Separator4.instance">
                        <attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
                        <attr name="position" intvalue="1200"/>
                    </file>
                    <file name="FileSystemAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.FileSystemAction"/>
                        <attr name="position" intvalue="1300"/>
                    </file>
                    <file name="Separator5.instance">
                        <attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
                        <attr name="position" intvalue="1400"/>
                    </file> 
                    <file name="ToolsAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.ToolsAction"/>
                        <attr name="position" intvalue="1500"/>
                    </file> 
                    <file name="PropertiesAction.instance">
                        <attr name="instanceClass" stringvalue="org.openide.actions.PropertiesAction"/>
                        <attr name="position" intvalue="1600"/>
                    </file> 
                </folder>            
            </folder>
        </folder>
    </folder>
</filesystem>

Don't forget to prepare all these resource files that registered in above layer.xml, such as scala16x16.png etc

After that, write an one-line service descriptor org.openide.filesystems.MIMEResolver under META-INF/services, which looks like

org.netbeans.modules.scala.editing.ScalaMimeResolver

That's it.

Posted at 12:59PM Mar 21, 2008 by dcaoyuan in NetBeans  | 

Fortress for NetBeans Screenshot#2: Mark Occurrences and Instant Rename

Well, I'm going on Fortress for NetBeans, and quickly get some basic Mark-Occurrences working, and of course, Instant-Rename .

As more thoughts are proved, I may begin new Scala editor module soon.

Posted at 05:36PM Mar 20, 2008 by dcaoyuan in NetBeans  | 

Where We Are - Stock Marketing of China

I bought a stock in Dec 7, 2007, which was delisted soon, till now, that's why I don't care recent down of China stock markets.

But it seems the stock I bought is near to be listed again, so, I did some quick computing, and here's a summary prediction of SSE Composite Index (000001.ss):

The index will bounce to about 5600 in April, then down to about 4000 at June/July.

But I'll be very care to rely on this prediction, because it's a quick result (a carefully result needs a lot of computing time) and looks like really no-directions in the near future.

Posted at 03:27PM Mar 20, 2008 by dcaoyuan in Trading  |