After many failed attempts at building webkit I was finally able to get all the pieces together and build and debug webkit under visual studio. I can now start playing with the live system, rather than browsing through the code.

The big problem I faced was the builds failing because the line endings used in the script files were set to CRLF whereas I had setup my cygwin to use unix line endings. I was able to use Notepad++ to fix them and finally get a working build from the latest source. Another issue I faced was that using subversion from within cygwin was very error prone and always left the working copy in some errorneous state, using tortoise svn directly from windows helped here.

So far the webkit code base looks a lot better organized than the mozilla code base so hoping that I can learn and contribute back to this project.

When I had first read about CMM I had thought that it was the holy grail that all software companies should be working towards, at that time I was still in school and idealism took precedence over reality.

After having worked in this field for some years now, I feel that CMM is so far removed from reality that it’s a joke now. But still services companies in India continue to tout CMM capability as a selling point. Infact India has 75% of the world’s CMM Level 5 companies. But the sad truth is that a lot of these certifications have been issued by bribing, wining and dining the certifying authorities. As far as I know only the top IT firms actually enforce CMM in actual work, for other companies CMM processes are only followed when there is a process audit

So anyways I read this article by James Bach today, which had been published in Scientific American way back in 94 which is titles “The Immaturity of CMM”, if you feel that CMM is great then I think this is one piece that you should read to get a balanced view.

Powered by ScribeFire.

I got back to application development using Java after spending the last 4 years developing applications using .Net and C++ and things have really changed!!! I have spent the last one month trying to catch up on all the new stuff in the Java world. Updating myself with the latest changes to the syntax only took one day though and I was surprised to see that in the latest incarnation Java 5 and .Net are like twins. If you don’t believe me then read this.

So after updating myself with the platform changes I started going through the technology stack, it’s been a month and I think I have only scratched the surface of whats out there and I’ve been only looking at enterprise application development technologies, it actually took me a week to review all the web application development options! So far here’s what I’ve looked at, as in, read through the reference manuals, installed and played around a bit, Hibernate, JPA, JDBC, Spring, Struts, EJB, Velocity, Tapestry, WebWorks, Cocoon, Log4j, JBoss, Tomcat, JNDI, JMS, NetBeans, Eclipse, Jetty, Glassfish.

Today while I was trying to read the official docs for Java EE, I spent half an hour on the Sun site trying to decide what I wanted to download, did I want Java SE 6 or Java EE 5 Well after finally finding what I wanted, I went for a coffee break and for some reason the dialogue between Neo and the Architect from “Matrix Reloaded” kept popping up in my mind. I think the connection I was making is that the Java eco-system has become extremely complex, not unlike the stage C++ had reached before the advent of Java, at that time Java was “Neo”.

So who’s the new “Neo”? I really can’t predict, but it’ll be simple to use, have less configuration, it just works defaults, easy to get started, eventually have good performance and most importantly have a passionate user community, that will keep improving on it and ultimately cause it to become the next Java . Looking at todays options, I think “Ruby” seems to meet this criteria the most. You never know 5 years down the line we could all be thinking about how it took weeks to setup a simple application using the Java platform and how going through all those xml files and annotations gave you a headache.

I’ve included the entire dialogue between Neo and the Architect below, substitute the Matrix for Java and Neo for Ruby and njoy!

My favourites, keeping in mind the above context

• The problem is choice – Hell Yeah!!!!
• Denial is the most predictable of all human responses. – Which is what a lot of people who read this will be feeling
• As you are undoubtedly gathering, the anomaly’s systemic – C++ and now Java

The Architect – Hello, Neo.

Neo – Who are you?

The Architect – I am the Architect. I created the matrix. I’ve been waiting for you. You have many questions, and although the process has altered your consciousness, you remain irrevocably human. Ergo, some of my answers you will understand, and some of them you will not. Concordantly, while your first question may be the most pertinent, you may or may not realize it is also irrelevant.

Neo – Why am I here?

The Architect – Your life is the sum of a remainder of an unbalanced equation inherent to the programming of the matrix. You are the eventuality of an anomaly, which despite my sincerest efforts I have been unable to eliminate from what is otherwise a harmony of mathematical precision. While it remains a burden to sedulously avoid it, it is not unexpected, and thus not beyond a measure of control. Which has led you, inexorably, here.

Neo – You haven’t answered my question.

The Architect – Quite right. Interesting. That was quicker than the others.

*The responses of the other Ones appear on the monitors: “Others? What others? How many? Answer me!”*

The Architect – The matrix is older than you know. I prefer counting from the emergence of one integral anomaly to the emergence of the next, in which case this is the sixth version.

*Again, the responses of the other Ones appear on the monitors: “Five versions? Three? I’ve been lied too. This is bullshit.”*

Neo: There are only two possible explanations: either no one told me, or no one knows.

The Architect – Precisely. As you are undoubtedly gathering, the anomaly’s systemic, creating fluctuations in even the most simplistic equations.

*Once again, the responses of the other Ones appear on the monitors: “You can’t control me! F*ck you! I’m going to kill you! You can’t make me do anything!*

Neo – Choice. The problem is choice.

*The scene cuts to Trinity fighting an agent, and then back to the Architect’s room*

The Architect – The first matrix I designed was quite naturally perfect, it was a work of art, flawless, sublime. A triumph equaled only by its monumental failure. The inevitability of its doom is as apparent to me now as a consequence of the imperfection inherent in every human being, thus I redesigned it based on your history to more accurately reflect the varying grotesqueries of your nature. However, I was again frustrated by failure. I have since come to understand that the answer eluded me because it required a lesser mind, or perhaps a mind less bound by the parameters of perfection. Thus, the answer was stumbled upon by another, an intuitive program, initially created to investigate certain aspects of the human psyche. If I am the father of the matrix, she would undoubtedly be its mother.

Neo – The Oracle.

The Architect – Please. As I was saying, she stumbled upon a solution whereby nearly 99.9% of all test subjects accepted the program, as long as they were given a choice, even if they were only aware of the choice at a near unconscious level. While this answer functioned, it was obviously fundamentally flawed, thus creating the otherwise contradictory systemic anomaly, that if left unchecked might threaten the system itself. Ergo, those that refused the program, while a minority, if unchecked, would constitute an escalating probability of disaster.

Neo – This is about Zion.

The Architect – You are here because Zion is about to be destroyed. Its every living inhabitant terminated, its entire existence eradicated.

Neo – Bullshit.

*The responses of the other Ones appear on the monitors: “Bullshit!”*

The Architect – Denial is the most predictable of all human responses. But, rest assured, this will be the sixth time we have destroyed it, and we have become exceedingly efficient at it.

*Scene cuts to Trinity fighting an agent, and then back to the Architects room.*

The Architect – The function of the One is now to return to the source, allowing a temporary dissemination of the code you carry, reinserting the prime program. After which you will be required to select from the matrix 23 individuals, 16 female, 7 male, to rebuild Zion. Failure to comply with this process will result in a cataclysmic system crash killing everyone connected to the matrix, which coupled with the extermination of Zion will ultimately result in the extinction of the entire human race.

Neo – You won’t let it happen, you can’t. You need human beings to survive.

The Architect – There are levels of survival we are prepared to accept. However, the relevant issue is whether or not you are ready to accept the responsibility for the death of every human being in this world.

*The Architect presses a button on a pen that he is holding, and images of people from all over the matrix appear on the monitors*

The Architect – It is interesting reading your reactions. Your five predecessors were by design based on a similar predication, a contingent affirmation that was meant to create a profound attachment to the rest of your species, facilitating the function of the one. While the others experienced this in a very general way, your experience is far more specific. Vis-a-vis, love.

*Images of Trinity fighting the agent from Neo’s dream appear on the monitors*

Neo – Trinity.

The Architect – Apropos, she entered the matrix to save your life at the cost of her own.

Neo – No!

The Architect – Which brings us at last to the moment of truth, wherein the fundamental flaw is ultimately expressed, and the anomaly revealed as both beginning, and end. There are two doors. The door to your right leads to the source, and the salvation of Zion. The door to the left leads back to the matrix, to her, and to the end of your species. As you adequately put, the problem is choice. But we already know what you’re going to do, don’t we? Already I can see the chain reaction, the chemical precursors that signal the onset of emotion, designed specifically to overwhelm logic, and reason. An emotion that is already blinding you from the simple, and obvious truth: she is going to die, and there is nothing that you can do to stop it.

*Neo walks to the door on his left*

The Architect – Humph. Hope, it is the quintessential human delusion, simultaneously the source of your greatest strength, and your greatest weakness.

Neo – If I were you, I would hope that we don’t meet again.

The Architect – We won’t.

The build is the pulse of any software development activity and a good build system facilitates quality software development. My current thoughts on a build process are as follows

Build Machine

The build machine is a dedicated physical or virtual machine whose sole purpose is to build your product. It should not be used for development or QA activities. Here are some points to keep in mind when setting up the build machine.

• Use a physical or virtual build machine depending on how often you plan to build and whether the build is initiated manually or automatically. A virtual machine has the advantage that it can do the build, copy the files to a file server and then it can be reset to it’s initial clean state. Also a virtual machine does not require an expensive machine that will be idle most of the time. On the other hand if the build is initiated automatically or happens multiple times during the day, it is best to use a dedicated physical machine.
  • Another possibility is to use one Virtual Server that hosts build machines for multiple projects, this way the physical hardware is used optimally.
• Keep the number of softwares installed on the build machine to a minimum.
• Should have ample storage space, if the builds will also be archived on the build machine itself.
• Build machine should be fast to keep build times low. This is especially important in C++ like projects where typical build times can be 30+ minutes. Also having a fast build process means that the time from a checkin to a build pass/fail result is minimized thus ensuring that time is not wasted waiting for the build results.
• Have some sort of remote desktop capability installed on the machine. This proves to be invaluable when you need to do emergency builds from home at night.

Process

The build process is more than just a compilation of the code that is done on a developer’s workstation, a good build process is able to generate the final set of artifacts that is sent to the user in one step. For e.g. a build process for a desktop application would build the installer, documentation, licenses etc. for the entire product, in case of a web application the build will compile the code files, supporting assets and deploy to a test or staging server. A good build system should be configurable and able to build multiple editions of the product.

There are a lot of options available for build tools, ranging from the simple make files and batch files to continuous integration systems like CruiseControl, in between are the new age build systems like Ant, NAnt, MSBuild etc.

Here are some tasks that need to be done when creating a build system. Not all of them will be relevant to all projects and there will be some steps that are specific to individual projects. Also the specifics on how to achieve the tasks will be different based on the build tool you use.

1. Clean up/Create the folders where the project output will be generated.
2. Get the latest sources from source control.
3. Update the build numbers in code and documentation.
4. Tag the code in source control.
5. Build the code.
6. Run automated unit tests.
7. Run automated functional tests.
8. Run automated regression tests.
9. Build the documentation.
10. Generate release notes
11. Build the installers.
12. Copy the artifacts to a folder for archiving or to the machine that is used for archiving the builds. Make sure that the folder are named appropriately.
13. Deploy the product to a test server.
14. Send out build status email.
15. Clean up and temp files and folders.

The following sections talk in detail about some of the tasks mentioned above

Build numbers

The build number should be of the format major.minor.patch.build (Note that this is slightly different from the .Net recommendations)

major and minor revision numbers are generally decided by the product marketing. The patch and build numbers are generally used by the engineering team to identify the product.

The build number should be a monotonically increasing sequence across your mainline code and is independent of the major and minor version numbers. For a branch the build number should remain constant and the patch version should increment with each labeled build, the labeled builds on a branch would usually correspond to updates to already released products, which is why the major minor and build numbers remain the same.

For e.g.

Mainline – 1.0.0.100, 1.0.0.101 … 1.5.0.125 … 2.0.0.180

In the above case the build number starts off with 100 and each subsequent labeled build of the mainline has the next higher build number, this remains true even as the major and minor version numbers progress to 1.5 and 2.0

Branch – 1.5.0.125, 1.5.1.125, 1.5.2.125

In the above case we create a branch for the 1.5.0.125 build which is a release branch. Any subsequent bug fixes/minor changes to this branch get labeled with increasing patch numbers while the major, minor and build numbers remain constant, this makes it clear that the builds are just patch updates to the release.

There are many alternatives to the above scheme for build numbering, one of them is major.minor.build.patch, the problem with this one is that the user facing version number of the product major,minor and patch are not contiguous, in case of major.minor.patch.build, we can simply drop off the last part of the version string and get the user facing version string major.minor.patch

Tagging code

If a labeled build is being made as opposed to a test build, then the code should be tagged in source control. This usual convention is to just tag the code as Build_ for e.g. Tags/Build_1.5.0.125

When a build is being released to the customer, then in addition to the tag, a branch should also be created for the build on which any subsequent bug fixes and minor changes can be made without affecting the mainline. for e.g. Branches/Release_1.5.0.125

Build the code

When building the code, consider building both the Debug and Release versions of the code, the Debug versions are useful when you have to debug a particular version of the product at a later stage. Although the code can always be retrieved from source control, it can be time consuming in case the code base size is big.

Automated tests

If possible we should run automated tests as soon as the build is made, a failing test should result in a failed build. Most build tools have integration with unit testing tools like NAnt, Ant etc. also batch files and other custom executables can always be used to invoke the test scripts.

Building documentation

If the product includes documentation then it should also be built along with the build, further the documentation should be labeled as well indicating the version of the product for which the documentation is written. Again standard tools like Microsoft Help Compiler, NDoc, SandCastle can be used to generate the documentation.

Release notes

Release notes should be automatically generated for each build, this can be done by either pulling in the list of changes from the source control or by pulling in the task ids that have been implemented since the last build from whatever issue tracking system you are using.

Building installers

For most products a installer will be required that will deploy the product on the users machine/server.

Some of the options for building installlers are

1. Installshield - Very costly and bulky.
2. Wise - Less expensive and not so bulky.
3. NSIS- Free, but not as many features as Installshield and Wise
4. Wix - Microsoft has open sourced the toolkit that allows generation of MSI files from xml.

Whatever tool is used, this should be integrated into the build system so that as a result of a successful build the installer is created as well.

Build archiving

Every labeled build that is created should be archived on some machine so that it can be used at a later stage for testing and debugging.

References

Some good references for build processes and configuration management

• High-level Best Practices in Software Configuration Management
• SCM Patterns for Agility
• Streamed Lines: Branching Patterns for Parallel Software Development
• Team Development with Visual Studio .NET and Visual SourceSafe

Recently I downloaded Picasa from Google to manage my huge collection of photographs, I was totally blown away by the great experience of using such a great piece of software. Everything was so intuit, and the UI was really slick and well organized. Being a developer the first thing that came to my mind was to see what toolkit they were using, I looked at the application directory to see if there were any obvious clues, not finding any I fired up the trusty Spy++ and found to my surprise that the whole Picasa UI is just one big window with a class name of “ytWindow”. This means that they custom draw everything i.e. use their own GUI toolkit.

Personally I would have never gone in for such a solution but now taking a look at how well the application has come out, I am definitely going to give this option a more thorough analysis.

In the process of finding the UI toolkit for Picasa I researched a lot of other UI toolkits. Seems like QT is a good option these days but is ridiculously expensive for anyone other than big enterprisey clients, GTK 2 seems to be the best among the open source alternatives. Also I read some good reviews on FLTK, might be interesting to take a look at it for developing prototypes.

Over the last year I’ve downloaded the source for many popular open source tools like Gaim, eMule, Mozilla etc. in order to read the code. One big problem I always faced coming from a Visual Studio background was the lack of a solution or workspace kind of concept that allows for easier code browsing. Today after reading a bit about ctags and wondering if I should give XEmacs yet another try I stumbled upon a cool new feature in VS 2005 “New project from existing code”, this is accessible via File->New->Project from existing code. It allows you to specify the folders where the source is available and also allows you to customize the build commands that are used, so you can actually use the makefiles that ship with the source to build rather than using VS. Intellisense in 2005 is damn good and now finally I can browse the code in an environment that I am very familiar with.

A few years back when I first started reading about design patterns, refactoring, OO principles etc. I got it into my head that I should “Always program to an interface not an implementation”. So whenever I needed to create a new class/abstraction I used to start of with an interface IThis, IThat etc. over a period of time I saw that most of my interfaces were only ever implemented by one class. This started becoming a pain to maintain as adding a new method meant that the method needed to be added to both the interface and the implementation class, further since there was only one implementation of the interface some developers started substituting one for the other and freely casting between the interface and the implementation. This became a source of constant code review changes and eyesores in code, with some methods expecting interfaces and others expecting the implementation class. Of course with strict code reviews we would never have got into this situation, but I’ve been working in startup companies all this time where getting something out there to earn $$$ has much higher priority than clean code.

Over the last few months I had to introduce some new features into our product which required that an abstraction that previously only had one implementation should now have 3, this was one of the cases where the class did not implement an interface. Contrary to what I had expected, refactoring the class and extracting an abstract class from it which was then subclassed by the different implementations lead to a very clean and satisfying design. This experience has taught me that unless you are writing code that interacts with something that is not under your control (a component or library for e.g.) it’s never a good idea to start with an interface.

On the other hand another feature that required extending an interface that was already defined became a big mess as a lot of the methods did not have a reasonable implementation under the new scheme and had to throw NotImplementedException’s. It was also difficult to refactor the interface as there were a huge number of dependencies. This could have been avoided if the Interface had been extracted on a need to need basis, further this would have lead to much more granular and cohesive set of interfaces rather than one huge bloated interface.

Starting with an interface on inward facing code means that you are making certain assumptions about how your code will evolve, in my experience these asumptions are almost always wrong and you end up supporting an interface that was never required in the first place. So now I start off with a concrete class and extract interfaces from the class as and when required, this has lead to much cleaner code and reduced the overhead of redundant definitions.

What have been your experiences in this matter?

An uber cool use of web services, Amazon has released the “Amazon Elastic Compute Cloud (Amazon EC2)” web service, it’s in limited beta right now.

Check it out.