Parsing Performance of Scala for NetBeans
I'm re-considering the indexing mechanism of Scala for NetBeans. The indexing is used to store meta-info of parsed templates/methods etc, for auto-completion and document/source offsets searching. Currently, the parsing phases include lexer, syntax and semantic analysis, not include type inference and type check.
With a basic performance testing on all Scala standard library and liftweb's library, the maxima parsing time seems less than 1s, the average parsing time is around 0.2s, not bad.
So, this may let the indexing feature a lot simple, I can store classes/objects/traits' meta-info only, instead of including their type parameters and their members (fields/methods, scoped importing etc), these additional information can be got via re-parsing the source file or querying the class file.
Posted at 08:30PM Jun 10, 2008 by dcaoyuan in NetBeans |
Bundled Latest Scala Runtime to Scala for NetBeans
I just bundled latest Scala runtime to Scala for NetBeans, the version is 2.7.1.final, this brings two things:
First, you do not need to set SCALA_HOME any more to get whole plugins working. But, you can still set SCALA_HOME to specifying the target Scala version, if so, you should also need to download and unzip source jars to $SCALA_HOME/src;
Second, I'll begin to write some code in Scala instead of Java for Scala plugins. I can evaluate the features of Scala plugins in daily work, find and fix more bugs of plugins.
Posted at 08:54AM May 27, 2008 by dcaoyuan in NetBeans | Comments[9]
Scala for NetBeans Screenshot#12: Better Completion with More Types Inferred
>>> Updated May 11
Mostly, infix expressions can be type inferred now. CTRL+Click on infix op name, it will bring you to declaration (since 1.0.29.0)
===
Well, the type inference work is not so easy (with performance in mind), but anyway, I've got a bit more progress, at least, the chained member call can now be correctly inferred in a lot of cases. It's some level as Tor's JavaScript for NetBeans now.
First, let's create a val "node", which is a "scala.xml.Node"
Then, input '.' to invoke completion, as I know which type is of "node", the proposal items look good.
I choose "descendant" function (which returns a "List"), and input '.' again, we can see the proposal items look still good.
These features also work on Java's class.
Known issues:
- It seems the indexing/scanning for Scala standard library source will perform twice when you first installed Scala plugins
- The type inference is not consistence yet, so don't be strange for the strange behavior sometimes
Again, don't forget to download scala standard library's source jars and unzip to $SCALA_HOME/src, per sub-folder per jar
Posted at 10:45AM May 09, 2008 by dcaoyuan in NetBeans | Comments[6]
Scala for NetBeans Screenshot#11: Go to Remote Declaration and Doc Tooltip
Two crazy days of my spare time, I was coding between sleeping and eating, with a lot of cups of coffee. Now Scala plugins support Go-To remote declarations (CTRL + Click on var/function name), and when you put cursor on the identifier name with CTRL pressed, or under auto-completion, the doc will also be shown as tooltip.
These features work for Java classes too with a bit poor performance, I'll fix it later (fixed).
Not all identifiers have been type inferred, so these features are not applicable for all identifiers.
Please update to Scala Editing module as version 1.0.26.xxx when it's available, which is the only stable one these days. Remember to unzip Scala lib's source under $SCALA_HOME/src
Posted at 12:04AM May 03, 2008 by dcaoyuan in NetBeans | Comments[8]
Scala for NetBeans Screenshot#10: Working on Auto-Completion for Java
I've done some hacking work to get Java classes to be completion enabled, but it's not full functional. Any way, it's a good start point for auto-completion for Scala plugins, I hope to get more type inference work to be finished, and finally support both Scala/Java classes smart-completion.
Click on the picture to enlarge it
Posted at 08:09PM Apr 29, 2008 by dcaoyuan in NetBeans | Comments[3]
Scala for NetBeans Screenshot#9: Working on Auto-Completion
With the indexed cache of project's Class/Trait/Object, and Scala standard library's source files, the auto-completion is a bit smarter now. If the val/var is defined with type, the auto-completion can know which methods will be suggested. (Not work for java classes yet)
To get this working, you should follow these steps:
- Update to newest Scala plugins (Editing version 1.0.21.1)
- Delete the old-cache files which are located at your NetBeans's configuration directory (for example, .netbeans/dev/var/cache).
- Download Scala standard library's source file, unzip them to $SCALA_HOME/src, per sub-folder per source jar file
Click on the picture to enlarge it
Posted at 07:40PM Apr 25, 2008 by dcaoyuan in NetBeans | Comments[4]
New Scala Plugins for NetBeans are Available for Public Test, and Fortress, Erlang
>>> Updated Apr 22
Due to the incompatibly changes of NetBeans underlaying modules, Scala plugins can not be installed/updated on NetBeans 6.1 any more, you should get the latest nightly build to play with Scala plugins.
===
>>> Updated Apr 19
- Fixed some broken syntax
- Fixed NPE caused by brace completion
- Fixed indentation of case class/object
- Added formatting options ("Preference" -> "Scala" -> "Formatting")
===
The new written Scala plugins for NetBeans are available for public test now, which can be installed on NetBeans 6.1 RC, and latest NetBeans nightly build. To get start, please visit http://wiki.netbeans.org/Scala
The following features are ready for test:
- Syntax highlighting
- Auto-indentation
- Brace completion
- Formatter
- Outline navigator
- Occurrences mark for local variables and functions
- Instance rename for local variables and functions
- Go-to-declaration for local variables and functions
- Scala project
- Basic debugger
And with known issues:
- Auto-completion it not fully supported yet and not smart
- There is no parsing errors recovering yet
- Semantic errors are not checked on editing, but will be noticed when you build project
- Due to the un-consistent of Scala's grammar reference document, there may be some syntax broken issues
BTW, Fortress editing plugin is also available on "Last Development Build" update center, see the installation part of http://wiki.netbeans.org/Scala to get it installed. It's a very alpha stage plugin.
And, Erlang plugins are also available from "Last Development Build" update center too, that is, you can install and use Erlang plugins with Ruby, Scala, JavaScript on the same NetBeans IDE (6.1 RC or nightly build). Thanks to Tor's work, the indexing performance has been improved a lot.
Erlang plugin will be rewritten in the near future too.
Posted at 09:21AM Apr 18, 2008 by dcaoyuan in NetBeans | Comments[13]
Scala for NetBeans Screenshot#8: Working on Indexer
I've done some basic indexer code, that is, all source files under a project will be parsed, analyzed, then indexed (class/object/trait, functions, fields etc). But it's just a start, before I finished type inference, if you press CTRL+SPACE to invoke completion, there are a lot of indexed Class/Object/Trait/Function will be roughly shown on you :-), it's not smart, it's more like a puzzle, you should decide which one is applicable by yourself. But you can get a view of the coming completion feature.
Click on the picture to enlarge it
Posted at 03:08PM Apr 14, 2008 by dcaoyuan in NetBeans | Comments[4]
Progress of Scala for NetBeans - with a New Written Lexer and Parser
According to this post:
I talked to Jetbrains about this, and they told me that they stopped working on the Scala plugin for the time being, because - demand for Groovy/Ruby was higher - the language was moving too fast - Scala is a terribly difficult language for compiler/tool writers, and the only good way to analyze Scala programs might be through the official compiler, which didn't yet support this
It's true that "Scala is a terribly difficult language for compiler/tool writers", but I'm trying to bypass "the only good way to analyze Scala programs might be through the official compiler"
Before rewriting Scala for NetBeans, I considered some parser choices, one was Scala's native compiler, which is good for compiling/building Scala project, but not suitable for Editor. And JavaCC, ANTLR, which may be good enough, but it's not natural to express Scala's grammar.
Then I found Rats! which is used by Fortress, a very very clean, powerful parser generator. After couple of days working, I got an incremental lexer for Scala, and a parser for Scala that with Scala's grammar being naturally expressed (the grammar definition is ParserScala.rats). The benefit of a complete controllable parser is that I can now do some type inference and wholly semantic analysis freely and immediately.
Another progress is that I've decoupled the Scala project's dependency on Java.source's classpath in NetBeans, instead, GSF's classpath is used in Scala project module now. That means, I can begin the indexer for Scala's standard library and project source files.
The next steps will be type inference; smart completion with type inferred information; indexer for later refectory and usages searching; parsing error recover etc.
Posted at 07:20AM Apr 13, 2008 by dcaoyuan in NetBeans | Comments[4]
Where We Are - Rewriting Scala for NetBeans
>>> Updated:
The rewritten plugin will be available, depending on the NetBeans' nightly build, it may be available after 10 hours or more. New features include a formatter (CTRL+SHIFT+F), and better brace completer. Smart auto-completion does not work now, needs further working.
===
The new Scala for NetBeans is going to form a good shape, I've got syntax highlighting, indentation, formatting, brace matching, and basic structure outline working. Here's a snapshot showing complex Scala statements being highlighted and reformatted properly:
Posted at 01:58PM Mar 31, 2008 by dcaoyuan in NetBeans | Comments[1]
Begin Rewriting Scala for NetBeans
I'm re-writing Scala for NetBeans. Everything is broken except syntax highlighting.
Please be patient to wait for further progress.
Posted at 10:30PM Mar 23, 2008 by dcaoyuan in NetBeans | Comments[2]
Developing IDE Based on GSF for NetBeans#1 - Minimal Support
There has been GSF (Generic Scripting Framework) which is Tor's working derived and abstracted from Java supporting code, and, the base of Ruby/JavaScript support for NetBeans.
So, how to develop an IDE based on GSF for NetBeans? I'd like to share some experiences in this series of articles, a series of outline description, without too much code and details, for detailed information, please go into the source code on hg.netbeans.org
I. Minimal Support - Highlighting
To implement a minimal support of your editor, you need to implement/extend following classes/interface:
- public class ScalaLanguage implements GsfLanguage
- public class ScalaMimeResolver extends MIMEResolver
- public enum ScalaTokenId implements TokenId
- public class ScalaLexer implements Lexer<ScalaTokenId>
Then, register your language in layer.xml:
<filesystem>
<folder name="Editors">
<folder name="text">
<folder name="x-scala">
<attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>
<file name="language.instance">
<attr name="instanceCreate" methodvalue="org.netbeans.modules.scala.editing.lexer.ScalaTokenId.language"/>
<attr name="instanceOf" stringvalue="org.netbeans.api.lexer.Language"/>
</file>
<folder name="FontsColors">
<folder name="NetBeans">
<folder name="Defaults">
<file name="coloring.xml" url="fontsColors.xml">
<attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>
</file>
</folder>
</folder>
</folder>
<folder name="CodeTemplates">
<folder name="Defaults">
<file name="codetemplates.xml" url="codetemplates.xml">
<attr name="SystemFileSystem.localizingBundle" stringvalue="org.netbeans.modules.scala.editing.Bundle"/>
</file>
</folder>
</folder>
<folder name="Keybindings">
<folder name="NetBeans">
<folder name="Defaults">
<file name="org-netbeans-modules-scala-editing-keybindings.xml" url="DefaultKeyBindings.xml"/>
</folder>
</folder>
</folder>
</folder>
</folder>
</folder>
<folder name="GsfPlugins">
<folder name="text">
<folder name="x-scala">
<file name="language.instance">
<attr name="instanceOf" stringvalue="org.netbeans.modules.gsf.api.GsfLanguage"/>
<attr name="instanceClass" stringvalue="org.netbeans.modules.scala.editing.ScalaLanguage"/>
</file>
</folder>
</folder>
</folder>
<folder name="Loaders">
<folder name="text">
<folder name="x-scala">
<attr name="SystemFileSystem.icon" urlvalue="nbresloc:/org/netbeans/modules/scala/editing/resources/scala16x16.png"/>
<attr name="iconBase" stringvalue="org/netbeans/modules/scala/editing/resources/scala16x16.png"/>
<folder name="Actions">
<file name="OpenAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.OpenAction"/>
<attr name="position" intvalue="100"/>
</file>
<file name="Separator1.instance">
<attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
<attr name="position" intvalue="200"/>
</file>
<file name="CutAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.CutAction"/>
<attr name="position" intvalue="300"/>
</file>
<file name="CopyAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.CopyAction"/>
<attr name="position" intvalue="400"/>
</file>
<file name="PasteAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.PasteAction"/>
<attr name="position" intvalue="500"/>
</file>
<file name="Separator2.instance">
<attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
<attr name="position" intvalue="600"/>
</file>
<file name="NewAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.NewAction"/>
<attr name="position" intvalue="700"/>
</file>
<file name="DeleteAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.DeleteAction"/>
<attr name="position" intvalue="800"/>
</file>
<file name="RenameAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.RenameAction"/>
<attr name="position" intvalue="900"/>
</file>
<file name="Separator3.instance">
<attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
<attr name="position" intvalue="1000"/>
</file>
<file name="SaveAsTemplateAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.SaveAsTemplateAction"/>
<attr name="position" intvalue="1100"/>
</file>
<file name="Separator4.instance">
<attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
<attr name="position" intvalue="1200"/>
</file>
<file name="FileSystemAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.FileSystemAction"/>
<attr name="position" intvalue="1300"/>
</file>
<file name="Separator5.instance">
<attr name="instanceClass" stringvalue="javax.swing.JSeparator"/>
<attr name="position" intvalue="1400"/>
</file>
<file name="ToolsAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.ToolsAction"/>
<attr name="position" intvalue="1500"/>
</file>
<file name="PropertiesAction.instance">
<attr name="instanceClass" stringvalue="org.openide.actions.PropertiesAction"/>
<attr name="position" intvalue="1600"/>
</file>
</folder>
</folder>
</folder>
</folder>
</filesystem>
Don't forget to prepare all these resource files that registered in above layer.xml, such as scala16x16.png etc
After that, write an one-line service descriptor org.openide.filesystems.MIMEResolver under META-INF/services, which looks like
org.netbeans.modules.scala.editing.ScalaMimeResolver
That's it.
Posted at 12:59PM Mar 21, 2008 by dcaoyuan in NetBeans |
Fortress for NetBeans Screenshot#2: Mark Occurrences and Instant Rename
Well, I'm going on Fortress for NetBeans, and quickly get some basic Mark-Occurrences working, and of course, Instant-Rename .
As more thoughts are proved, I may begin new Scala editor module soon.
Posted at 05:36PM Mar 20, 2008 by dcaoyuan in NetBeans |
Fortress for NetBeans Screenshot#1: Syntax Highlighting
>>> Updated Mar 19:
The outline navigator will do some unicode rendering; Brace matching works.
===
I'm planning to re-write Scala editor module, based on a new parser (maybe Tor from Sun and Carl from Google will help this project too). But before that, I'd like to prove some thoughts. That comes Fortress for NetBeans.
To know more about Fortress, please go to http://projectfortress.sun.com/Projects/Community/
Fortress's grammar is expressed in PEG (Parsing Expression Grammars), and is implemented on Rats!, a clean PEG parser generator.
I got some interesting experiences on how to let Rats! become a simple lexical token stream generator, of course, it has some limits, but these lexical tokens are used only for basic/quick highlighting, brace-matching etc, I do not use it for syntax parsing and semantic analysis, so the limits are not an effect. The syntax parsing, semantic analysis and grammar checking will be from original Fortress parser.
Here's a screenshot of my experimental work:
Actually I've done a bit of semantic works too, thus you can see the function definitions were bold highlighted and the outline navigator window was almost there.
Posted at 07:27PM Mar 16, 2008 by dcaoyuan in NetBeans | Comments[1]
ErlyBird 0.16.0 Released - An Erlang IDE based on NetBeans
I'm pleased to announce ErlyBird 0.16.0, an Erlang IDE based on NetBeans. This is an important feature release in size of 25M. If you have latest NetBeans nightly build installed, you can also install ErlyBird modules via update center.
CHANGELOG:
- Project metadata file is changed, please see Notes
- Instant rename (put caret on variable or function name, press CTRL+R, when rename finished, press ENTER)
- Go-To-Declaration to macros that are defined included header files
- Fixed: Go-To-Declaration to -inlcudelib won't work again after this include header file was opened in editor once
- Fixed: syntax broken for packaged import attribute
- Fixed: syntax broken for wild attribute
- Completion suggestion will not search other projects
- Track GSF changes, reindex performance was improved a lot; Can live with other GSF based language support now (Ruby, Groovy etc)
Java JRE 5.0+ is required.
To download, please go to: http://sourceforge.net/project/showfiles.php?group_id=192439
To install:
- Unzip erlybird-bin-0.16.0-ide.zip to somewhere.
- Make sure 'erl.exe' or 'erl' is under your environment path
- For Windows user, execute 'bin/erlybird.exe'. For *nix user, 'bin/erlybird'.
- Check/set your OTP path. From [Tools]->[Options], click on 'Erlang', then 'Erlang Installation' tab, fill in the full path of your 'erl.exe' or 'erl' file. For instance: "C:/erl/bin/erl.exe"
- The default -Xmx option for jvm is set to 256M, ErlyBird now works good with less memory, such as -Xmx128M. If you want to increase/decrease it, please open the config file that is located at etc/erlybird.conf, set -J-Xmx of 'default_options'.
When run ErlyBird first time, the OTP libs will be indexed. The indexing time varies from 10 to 30 minutes deponding on your computer.
Notes:
- Since project metadata format is changed, to open old ErlyBird created project, you should modify project.xml which is located at your project folder: nbproject/project.xml, change line:
<type>org.netbeans.modules.languages.erlang.project</type>
to:
<type>org.netbeans.modules.erlang.project</type>
- If you have previous version ErlyBird installed, you should delete the old cache files which are located at:
- *nix: "${HOME}/.erlybird/dev"
- mac os x: "${HOME}/Library/Application Support/erlybird/dev"
- windows: "C:\Documents and Settings\yourusername\.erlybird\dev" or some where
The status of ErlyBird is still Alpha, feedbacks and bug reports are welcome.
Posted at 06:24AM Mar 06, 2008 by dcaoyuan in NetBeans |