Soapbox Rants and Raves

I recently wrote a post about how Eclipse helps to maintain the developer’s focus; what I really wanted to write about was the Mylyn plug-in.  I recently worked on a small project where we setup a completely open-source development environment, built on top of an Ubuntu Linux server. What made the environment interesting, was how well everything worked together. One of the more challenging issues working on a small project is the communication and task management; this is where Mylyn helps smooth out the process.

Mylyn is a simple Eclipse plug-in that integrates with a large number of commercial and open-source issue tracking systems. On the surface, it looks like any other tracking tool, but provides far more functionality than a simple web-based solution.  Building on my previous post, Emacs to Eclipse, the developer can conveniently manage their programming assignments without leaving Eclipse. It even has a notification system which alerts the developer to newly created or assigned tasks. I believe this approach is far more effective than relying on email or external applications to manage the development process.

What makes Mylyn unique, is that each task can be assigned a context. The context represents the state of the Eclipse session, maintaining a list of all of open files. The first step is for the developer to activate a task. Once activated, the developer has the ability to assign a context to that task.  From this point forward, as long as that task is active, the context will be kept in sync with all of the files that the developer opens.  If the developer chooses to activate a different task, the current task is deactivated and the new task is activated. The open files are automatically closed and the files linked with the new task are opened; even restoring the cursor position within the files.  The is extremely helpful when working on multiple, unrelated tasks which are being worked in parallel.

There are a variety of other subtle integrations that also improve the developer’s effectiveness:

• One of the more convenient features is the ability to create tasks from any “warning” or “TODO” comment. The developer just right clicks on the marker to generate the new task.
• Mylyn is also integrated with the Team Plug-in (SVN/CVS). It will automatically populate the commit message with the task information.
• Using the plug-in’s file upload functionality,  developers can easily attach supporting documentation (stack traces or generated files). There is even a built-in screen capture utility which allows the developer to grab, crop, and annotate images, which can be quickly attached to the task.
• Mylyn magically tracks the amount of time the developer spends working on a task. It appears to be the real time spent on the ticket (based on having Eclipse open and active, not simply based on when the ticket was opened and closed; pretty cool!). This could be very useful in Agile projects, comparing the estimated time to the actual time spent on the task. What a great way to help the developer to improve their future estimations.

The primary issue tracking tool where I work does not integrate with Mylyn. It would be nice if it did, but I don’t know how important that really is. I have been thinking about the integration of a secondary, fine grain task management tool into the environment that is more developer focused.  I actually think it would be interesting to have two issue tracking systems. The first system would be exclusively for the QA and production support teams, mainly to track defects discovered in testing or production related issues.  The second system would be used exclusively to manage the development process. Why create tens or hundreds of tickets in the real ticket tracking system, just for the development team to build the next release? The answer really depends on the scope or purpose of the ticket, but generally it is too much work to create task level assignments in real tracking system. I would really like the tickets in the development environment be more fluid and granular, actually representing the work that needs to be done, not an outdated, vague interpretation of what might happen during development cycle. If a create needs to be created for a developer to refactor a method or add a jUnit, no big deal.  If I’m using the real production ticketing system, I would need several levels of approvals, both from the business sponsor and test teams. Unfortunately, this impledes progress and the actual work being performed is of no interest to them anyway, it is just part of the normal development process. The bottom line is that I currently have no way to easily task or manage the development process or myself; it is currently done through email chains and some crude “TODO” comments in the code. Kind of a bummer when the technology exists, for free (minimal configuration time) and could actually be a huge benefit for the development teams. If you include the planning aspects of the tool, I could see this being a huge help for the project managers as well.

Unfortunately, because of the infrastructure requirements (mysql, tomcat, etc.) of the open source tools (bugzilla, trac, etc.), it is politically too difficult for teams to integrate in the corporate environment. I recently read about an interesting Mylyn connects that lets you store the tracking information in a relational database, rather than a full featured ticket tracking system. I have yet to explore this option, but it might open up the door for allowing me to utilize Mylyn, without the infrastructural overhead. I will keep you posted!

A recent manager of mine worked tirelessly to drill this concept into our team: Build, Package, Deploy. If you step back and think, it is a very simple, yet powerful paradigm. I work in an environment that has not fundamentally changed the way we deploy J2EE applications for at least, I am only guessing here, but could be close to 10 years. The group that managed the Internet defined a standard footprint, multi-site J2EE cluster, with all components (Servlet, EJB, etc.) deployed on the same instance. Everyone was given the same configuration and we ended up building a large collection of heavy, monolithic applications. The worst part was that each application was dependent on the specific J2EE container and hardwired to the deployment environment, making an extremely static, inflexible, unscalable solution. Many of us developer’s silently rebelled against the J2EE bandwagon and built simple Servlet-based applications; making sure to use as little as possible, from the provided J2EE container (no connection pools, or even JMS). We pushed to use a simple Servlet container, such as Tomcat or Jetty, but our pleas went unanswered. The default architecture was structured in such a way, that everyone created huge, highly coupled, monolithic web applications; a single WAR that did everything, from vending JSPs, to performing business functions, to accessing the database. Unfortunately, I too was brainwashed by the architecture and stopped thinking about how architecture could be done differently. Fortunately for me, that recent manager was new to the company and had completely different ideas about architecture. Instead of starting a revolution, we embarked on a evolutionary journey to change the way we build software, rather than focusing on how the systems were deployed. One of the first points of enlightenment, to change the way we thought about building system: build it, package it, and then deploy it.

I will tie this paradigm back to the Lean principle of postponing decisions (committing to something) until you actually have to make the decision (the commitment). How does this apply here? When we build applications today, the developers know up front, where and how the system (code) will be deployed. The developer will make design decisions, many of which are unconscious, based on their knowledge of the deployment environment; hardwiring their implementation for what the application will look like when it is deployment. Without giving this much thought, it almost sounds like the right process. Unfortunately, we probably just boxed ourselves right into a corner!

Would it not be better, if the developers simply built components, the true business functionality, without thinking about how the software would ultimately be packaged or deployed? Obviously, there would be some architectural patterns to be followed, but the developer’s would have no idea how their component will be assembled into the final application. Now let’s imagine that we have a Component Packaging Specialist; they would be responsible for bundling up components into deployable packages. This person determines the optimal combination of components, creating a virtual application that performs the required business functionality. This virtual application is bundled in such a way as to create minimal dependencies between bundles, which are independently migrate-able and allow high scalability. Finally, the bundles are given over to a Deployment Specialist, who figures out how best to deploy the virtual application into the environment. Depending on the operational requirements, the virtual application could be deployed in any J2EE or simple Servlet container. How many of each bundles are to be deployed would be determined by the predicted transaction volumes. If the actual volume exceeds the expectations, the Deployment Specialist can easily adjust the environment to accommodate the volumes, without involving the development staff.

The primary point is to allow as much deployment flexibility as we can afford, by not letting the code dictate how or where the application will be deployed. This approach provides less risk (no need to change code) and creates fewer dependencies (better portability). Developers should remember and separate these three tasks as they design and implement systems, even if one person is performing all three roles. By separating these three tasks (actual disciplines), we should end up with better systems, providing the operational flexibility to meet the ever changing business demands.

I found the blog of Miško Hevery last week; he is an Agile Coach at Googe. Scanning his recent posts, he seemed to capture many of my own thoughts, saving me the time of actually having to put them into words! Please check out his posts, topics range from testability, to dependency injection, to Changing Developer Behavior(1)(2).

His blogroll lead me to some other great sites: The Agile Buddy and Agile Journal. I have yet to read these two posts, but they really grabbed my attention:

• Doing the Right thing vs Doing it Right
• Requirements Come Second

I’m sure that I will enjoy this read, as the second paragraph of the Requirements Come Second post states that  making the development environment more effective should be the first priority.  I have been a huge advocate of creating an effective development within teams for a long time. I typically feel that I’m on an island by myself, as most people don’t really care what or how something gets done, as long as the date is not missed.  Just getting it done has never been good enough for me.  Obviously, projects have to get done or none of us would have jobs. So the question is, why can’t we get the project and and do the right thing?  Hopefully, there will be some good wisdom in the post! Enjoy.