For.example

In this post I will cover my current work on Software Cartography. If you are unfamiliar with spatial-representation of software please refer to “A Software cartographer’s Vision” previously featured on this blog.

→ Codemap is available for download as Eclipse plug-in.

A major issue of software cartography is the base layout of software maps. A good base layout is both stable over time and conveys a meaningful grouping of software artifacts into islands (ie clusters). My initial attempt was to based the layout on vocabulary found in the source code, that is identifier names and comments. Vocabulary has the nice property that it is more stable over time than structure, and that it naturally conveys a meaningful clustering of latent topics.

However a first user study has shown that a vocabulary-based layout does not meet the developers intuition. Even though developers had been well aware that the layout was based on vocabulary, they took conclusions that assumed a structure-based layout. We learnt this from the think-aloud protocol used in the user study.

The user study included six developers that each had 1.5 hours time to explore an unknown software system. They were given six tasks of increasing complexity—cumulating in fixing an actual bug.

I will cover the user study in more details in a future blog post. In this post I shall discuss some proposals for alternative layouts…

Here is a preview of a recently submitted paper on Software Cartography. We report on preliminary results of an ongoing user study with both students and professional developers. Results are mixed and revised the assumption that lexical similarity is sufficient to layout the map. We are now working on a new distance metric that includes the ideal structural proximity proposed by the “Law of Demeter”. Also we are looking into new layout algorithms. For example, anchored MDS would allow developers to rearrange the map according to their system’s architecture.

NB if you are a professional developer from Switzerland, we encourage you to participate in our user study. Please contact David or me for more information.

The preview was created with Wordle, an idea that I owe to Tom Zimmermann.

The main focus of the SUITE workshop (co-located with ICSE) are search and software development. A frequently asked question is whether SUITE is limited to code search.

— SUITE is not limited to code search!

The general theme of SUITE is search-driven development. We consider search to be  a fundamental activity during software development. This includes (beside searching code, of course) search of external documents as well as search of runtime objects. Any search that may help when working with software.

Also, keep in mind that we are not only looking for solutions but also for studies of the problem space. So if you studied the search needs of developers, your results are as welcome as how to implement a new code search engine. The idea of SUITE is to explore all aspects of search in software development.

Lemme me give you an example that is not related to code only

• Assieme by Raphael Hoffmann etal is a meta-search engine for software documentation. Search results are presented as a browsable combination of API documentation, code examples and other information found on the web. Interestingly enough, code examples are not taken from source code repositories but from HTML pages. So you get a selection of examples that were considered worth to be put on the web by someone out there. All examples are compilable and thus ready to be paste in your program.

For further inspiration, you may find an exhaustive list of topics on the SUITE website—of course, submissions to the workshop are by no means limited to the topics covered there. If you have any position on software search, submission is open until Jan 19, 2010.

We are looking forward to hear your take on software search.

Be a rebel!

It is my vision that developers can speak of code as “up in the north”, “over in the west”, or “down-under in south”. I want to provide developers (and everyone else involved in software development) with a shared & stable & spatial mental model of their project.

A mental model of code that is shared with your team mates and that is stable over time and that is spatial so you can grok it. They way I try to reinforce this, is by providing a map in your IDE. The map is always visible in the bottom-left, just like the navi in your car. Whatever you do in the IDE is reflected on the map.

When you open a source file, its name pops up on the map.

When you browse references or callers, arrows show up on the map, pointing from where you are to references respectively callers.

When you run tests, tubes pop up filled with colored chemicals in green, yellow, or red.

When you are tracking down a bug in the debugger, arrows pop up and show the current stack trace.

The idea of the map is to provide you with a spatial model of your software. You will quickly learn that, for example, web UI code is in the north, the database layer in the south, unit tests are in the west, and the whole buggy XML mess over there in the east.

On the map, code is grouped by topic and not by structure. That is, even if your architecture sucks, you’ll get a meaningful map that will guide you out of the mess, as you refactor it.

At first, it can be unfamiliar to see code grouped by topic rather than by packages, but you’ll get used to it very quickly. Also, if you come back to the same project a year later, code is supposed to be roughly at the same location (unless of course, in the meantime, domain tectonics put the world of your application upside-down).

Cross-cutting topics are not always a sign of bad packaging. Just think of “session timeout” in both the user interface and the database layer. You won’t probably be able to factor it out into a common class. But still, codemap will put it on a landslide connecting north and south, such that you can find both classes in one place when working on that mysterious bug report related to timeouts. And even better, all your domain classes related to time will be grouped closely in the same neighborhood.

…and you will recall the blue avatar of your team mate that was busy down there yesterday. Dropping him a line of chat might be the quickest way to get that bug killed.

A prototype of Codemap is available for Eclipse.

Not everything is working yet up to our expectations, but you are welcome to give it a try and let us know how you like it. Just follow @codemap on twitter for news, feedback and questions.

— Adrian and David

Reverse Engineering is an art, some say. Not any more: Felix Leder recently presented RE-Google, a search-driven plugin for IDA Pro, that queries Google code search (GSE) for terms found in disassembled executables.

Google code search is very valuable when trying to find algorithms or code excerpts that contain this information. Often the few results you see on one page can already tell you what the function might be doing.

This plugin enumerates all functions and extracts strings, constants (also called immediate values), and the names of imported functions. If there is sufficient data, a Google Code search is performed and the result is added to the IDA database as function comment. Reviewing these comments sometimes turns the analysis of the considered function unnecessary and saves time.

trunk/src/lib/third-party/jsSHA/src/sha.js
FileZilla3/tags/3.0.9/src/putty/sshsh512.c
trunk/v1/puzzle/vista/js/encriptar.js
StepMania-3.9-src/src/crypto/CryptSH512.cpp
coreutils-6.4/lib/sha512.c

It seems to be very likely that the considered function is SHA-512 based on the results shown above. And it is :)

Tools like Felix’s plugin are the future of software engineering. Search-driven tools are not only useful for reverse engineering but also for forward engineering. As more and more code is made available on the interwebs, we need new ways to integrate this vast information in IDEs.

Note: if your are interested in some more reverse engineering, please check out No Starch press’s “The IDA Pro Book: The Unofficial Guide to the World’s Most Popular Disassembler” by Chris Eagle. A great read!

What I particular like about RE-Google is that it uses a “push” model. Search happens as a background activity and results are presented to the user as comments in the dissembled source code. No need to “pull” by typing queries and waiting for results.

At last year’s SUITE workshop we had been discussing proactive search tools as well:

• Code Conjurer by Olivier Hummel etal is integrated into Eclipse. In proactive mode, Code Conjurer suggests methods as you are drawing UML diagrams. Code Conjurer can suggest the typical set of methods offered by components matching the partial interface defined by the user.
• Rascal by Frank McCarey etal is also integrated in Eclipse. It recommends how and when to call the methods of objects from common libraries such as Java Swing, based on an analysis of existing classes. It uses collaborative filtering, similar to that used in
  online shopping sites, to recommend products on the basis of those similar customers bought.
• Code Broker by Yunwen Ye, maybe on of the first search-driven development tools, is integrated in Emacs. While developers work on their source code, Code Broker offers coding suggestions on the basis of information collected from similar components in the repository.

All these tools have in common that they monitor the user’s activities while he is working, and then offer information they considers useful in that specific context.

We hope to see even more proactive search tools at the next SUITE workshop, or maybe even HCI studies about their usefulness compared to old-school reactive search.

That being said, SUITE 2010 is looking for submissions!

See ya in Cape Town next May ;)

Stumbled upon RE-Google via @duritong, thanks!

Given that unit tests are the opposite of static typing, what is the opposite of test-driven design? Chris Smith’s great article on “What to know before debating type systems compares types” opposes types and tests with regard to correctness, performance and documentation. However, we’ve learned from the Agile community that test are most valuable when you let ‘em drive the design of your system.

Test-driven design, turned 10 this year and has been adopted by many major software companies. The benefits of TDD are well researched: 15-35% more time to release, but 60-90% less post-release defects. So well then, at the opposite of the spectrum, there must be type-driven design, right? Faster time to market but less error-prone?

Wait a moment!

In fact, recent research has shown that “the type system significantly turned out to increase the development time” (PDF). Bad news for Curry–Howard land.

Note: the linked PDF refers to the wrong workshop. In fact, the paper was published at PLATEAU not at VASE as the footer claims. Craig Anslow, organizer of both workshops told me that he made a mistake distributing the LaTex templates.

So, is there such a thing as type-driven design? If it exists, what is it good for? When does type-driven design shine? When does type-driven design break?

Pro

• It documents your system
• It forces you to keep documentation in coherence
• It forces you to do structured thinking.
• It is very good as long as you tackle the same old problems.

Cons

• If you start to do something new and creative it breaks down
• Its really really hard to innovate with types
• And it takes more time

Can we get the best of both worlds?

Synthesis

• Your language should be dynamic and not restrict your design with a type system.
• And after that, as an after-thought you go and add a type system

Pluggable Types to the rescue! Gilad Bracha (alas, he is not on Twitter) proposed optional type systems as a way to add a type system after your design has been stabilized. Optional types systems are pluggable, you can add and remove the type checks as fits your needs. This help to keep the type system out of your way while you explore the design of your system, and allows you to plug-in a custom type system once you’re done and know your system. Bracha mentions (among others) aliasing, ownership, and information flow. The choice of the checking is up to the developer.

“The real value of type systems isn’t the early error detection” — Gilad Bracha

Prototypes of a pluggable type system are available for Smalltalk and for Java. The Smalltalk prototype of pluggable types has been developped by Niklaus Haldiman, a former Master’s student of our research lab that is now with Google.

JSR 308, the Java prototype for pluggable types is being developed by Mahmood Ali and other at MIT’s CSAIL group. JSR 308 is built on top of Java’s annotation processor and likely to be included in Java 7. Mahmood demoed JSR 308 to me at ECOOP OOPLSA 2009 in Nashville, I was able to write a type system that checks several design pattern within a couple of hours only. You best check it our yourself. The JSR 308 folks are very responsive when you need help, just give it a try.

Alas, Newspeak does not yet feature pluggable types…

Another approach is Software Hardening, that is the gradual addition of more and more detailed type information as the design of your software is “hardening”, ie stabilizes. As you see, again the same concept as with Gilad’s pluggable types. A prototype of software hardening has been realized in the Thorn language, built by a team around Jan Vitek at Purdue University together with IBM research. Please refer to the Thorn language website for more information.

The STOP workshop at this year’s ECOOP features even more funky type systems, such as Philip Wadler’s blame calculus where they (as far as I understood, greek letters are not my field of expertise) keep track of type transition to be able to blame the right line of code when a dynamically typed program fails. They prove that one can shorten the chain of type transition to so called “threesomes”, which brings their approach into the reach of acceptable performance.

…but such suggestive concepts shall not be further discussed here, this is a serious blog after all :)

I joined twitter one and a half years ago, after having a hand written RSS feed for years. And gosh, I did not have the slightest idea to what kind of journey I was embarking! While my handwritten feed was limited to published papers, I soon learned to love the social aspect of twitter. At a sudden, folks would react to my news posts, like when I got in contact with Kent Beck regarding the code maps of JUnit. However, I have to admit that since the user explosion in 2009, twitter has become much less personal.

One of the success factors of twitter is that everyone can (and has to) find its personal usage model.

Here is how I use twitter for me and my research:

One of my motivations is to “maximize the value of your keystrokes”. For example, whenever I recommend a paper to a friend in private, I repeat this on twitter, etc. I first read of this on Jeff Atwood’s blog, but I guess the idea goes back to Jon Udell.

Often however, we cannot maximize our keystrokes because research 1.0 does not allow us to do so (or at least we fear that we cannot do so). In these cases I try to find solutions like publishing the wordle cloud of a paper under submission, which I have seen first being done on Tom Zimmermann’s blog.

A nice use case of twitter is to follow the hashtag of a conference. Alas most sci conference do not have good coverage (or none at all). There is e.g. not much point in tweeting from a talk without mentioning the speaker! In my experience technology and software conference get better coverage than research conference, but things started to change (see eg the recent MODELS/GPCE conference… or today’s one-man tour-de-force of realtime tweeting from the #archiPL workshop).

Of course, I also follow the tweets of over 200 researchers and practitioners (hey, the whole C2 crowd again united in one place!) Many of them new to me. Alas, since twitter broke replies, its much harder to stumble upon new folks.

Creating twitter accounts for your research projects. Both of my current research projects have their own twitter account. I did this after reading about the mars phoenix account by one NASA engineer, but stopped to blog in first person as if being the project after some time. In the beginning the project accounts were limited to updates and communication with users, but over time I started to post anything related to the project’s topic that I’d stumple upon (maximize the value of your keystrokes, again). One of the downsides that I see with splitting off these two accounts is that folks that visit my main account will not learn about my major fields of research, which I try to relax with occasional retweets. Some folks in our lab use the feed of their research project to post all SVN commit message. I am unsure about this practice, putting unfiltered streams from other sources on twitter does not seem to live up the full potential of this platform.

Recently I created a twitter account for a lecture that I coach as teaching assistant. I cannot yet tell where this leads, the lecture starts in February. However, I have used a blog in the last years and I assume it will lead me in a similar direction, ie posting links to additional reading material as well as useful tools and Eclipse plugins. Lukas and Erwann use twitter as well in the current software engineering lecture. However, their use is very specific to that specific lecture since the students are building a twitter client, and thus not suitable for other lectures.

I also created an account for our upcoming ICSE workshop on software search engines, but again I cannot yet tell where this will lead. We will certainly find better use than just posting deadlines…

Over time, my whole research group joined twitter as well, including Oscar. Most of them joined after I told them about maximizing keystrokes and how awesome following a conference is. (In the meantime, I even stopped being the most active twitterer in the group.) We found that the recent twitter list feature is a great way to present your research group on twitter. We created a new account for the group including lists for staff, alumni, students, projects, etc…

See you on twitter :)