The Next Holy War: Distributed VCS versus Centralized VCS

I often find myself being an advocate for something specific in the technology field. And, many times this relates to the common “holy wars” that most of us are familiar with. I sit on the emacs side of one, and the open source side of another. You can immediately understand what I’m referring to when I say “emacs”. The next topic for war I believe is going to come down to version control. If you want more proof, take a look at Derek Slager’s DVCS Myths post.
Currently there are two main types of version control and source management, centralized and distributed. I’ve used centralized systems for quite some time and although they have served a purpose…I find them to be outdated. Now, hear me out before getting frantic. The software industry is changing. We are in the midst of evolving the way we work. We want to work bigger, faster, better, stronger, right? There has been a push over the last 10-15 years to revamp our development methods. What I’m referring to here is the waterfall to agile approach of things.

The waterfall approach is about building very hard speced systems over the course of “x” to completion. Once the system is complete, you test, debug, etc. This is often very clunky, doesn’t describe the entire system, and takes an extremely long time to obtain any tangible assets. By this I mean that nobody can see a working system part of the way through. Thus, expectations are usually high leading let-downs as a result. The agile approach, in many ways, was created as a way to attack the problems at the core of the waterfall approach. Also, certain agile approaches can help do more such as improved development efficiency. With this in mind, and understanding that I sit on the agile side of that fence lets talk about source code management (SCM).

Centralized version control was fantastic. It provided the foundation for storing code in one place such that developers could work on code and then push it back into that one location. Not only was this able to happen, but we could lock, merge and tag. These are all good things by the way. What good would any SCM be without them. The lacking areas are the repository and the workflow.

Since we talked a bit about workflow above as it pertains to development, lets talk about how it relates to source management. In the likeness of the Pragmatic Programmers we know a few things to be good. That we want to refactor early and often. And, we do not ever want to settle with broken windows. These few things being said, lets discuss them as they pertain to workflow. In any SCM we can accomplish each, right? We checkout code, modify it, and commit new code. Centralized or decentralized. But, I’ve found it easier to manage the above “good things” within distributed version control system (DVCS). When I am working on refactoring code, I want to have an experimental branch. This allows me to create features, change API, and muck with the general internals of a codebase without destroying the greater good. That brings me to broken windows. If all developers commit to a central repository, then all code is based on said repository. Thus, if you have a team of n (including yourself), you can stop n-1 people from working for h number of hours in a day by commiting some code that you thought worked.

With a DVCS, you are commiting your code to a branch. This means that you are not commiting to the central repository. Sure you have to deliver your changes at some point, but you are more likely to have your code in working order. Also, it allows for easier testing of your code, since your code will be a subset of all new features and changes. So, when we refactor our code or write new features we are setting ourselves up better by having the DVCS.

Now that I’ve touched a bit on the workflow we can talk about the repository. Distributed version control systems are just that, distributed. The point is to make it easy for developers to make changes while being anywhere. The way this works is with distributed repositories. Distributed systems work by branching. Although the notion of branching is built into SVN and other centralized systems, they don’t work the same.

Distributed systems, as seen in the image, mean that everyone is a node, or using our terminology, a repository. Rather than just feeding off one main location, I can pull from my friends repository, push code changes up there or to a different repository. Branching allows for this sort of behavior.

So, when I modify code on my system and then commit it to my local repository, I can push those changes along to whomever or they can pull them down from me, and everyone has my latest changes.

So, I believe the thought going through your mind at this point is, wait…You’re contradicting yourself. If I want to commit often and refactor even more often then I’ll likely break something. Thats not good. The point here is that you shouldn’t work directly in your local repository. You should branch the repository and do work within the branch. For instance, say you are working on an OS. You can have a branch for networking. I can then grab that branch but if I have to implement some experimental IPV6 functionality, I’m not going to do so such that everyone gets those changes. I’ll branch the code base and do it there rather than expose that code to everyone. With a centralized system, you can branch the code…but merging the modifications back would be complicated and cumbersome, and in some cases not possible. If you need more proof take a look at Griddle Noise: What a DVCS gives me.
This being said, the industry uses centralized version control for most work currently. I feel that the change to agile programming should be accompanied by a SCM system that easily supports agile processes. A DVCS goes hand in hand with a distributed source management system.

There, I’ve stated my thoughts on the point. Flame away.

Prototyping in a Digital Age

I attended my first RefreshDC event this past Thursday. Pretty interesting I must say. For those unaware of the procedure; the venue is a room with projectors, seats, and a podium. Dude stands at the front and talks while taking questions. Clean, simple, and if kept to the timeframe…engaging.

Todd Warfel was the man presenting and his topic was prototyping. I’m a fan. I have to say he had some really interesting suggestions and ways to grab feedback. From using paper and colored cellophane to a miniature layout in html, he had a plethora of ideas. In my opinion that was where the most value came. His presentation talked of seven steps to take when prototyping for someone which were very generic and sounded like marketing babble, but when he started to talk between the points he provided a great deal of information.

The first think I want to make note of, because I feel it holds a great deal of relevance to what I do, is when to prototype. Todd mentioned that you should prototype early and often…I believe that he has this right. Early prototypes, especially when you are trying to gain requirements, is pivotal in a good product. You need to put something in front of people and find out what they like and dislike about something you plan to put time, sweat, and money into. Doing it any other way just doesn’t seem to make a lot of sense to me. Why build the requirements for a system when you haven’t thrown it at a user yet? You’ll spend significantly more to fix it later on in both branding and development work. Also, prototyping often allow you to focus your effort. You can parallelize the work of the next feature with the development of the last. And as with anything, the more you do it the better you get.

The next thing that I felt was a great takeaway was his mention of bringing in a good mediator. I don’t believe this was a point in his presentation but he mentioned it between points. In essence he mentioned that a mediator for user testing and prototyping should not try to help or suggest the answers to people. It’s generally very difficult to not help users when they are struggling, especially when you built the thing they are trying to use. However, if you really want good feedback on what does and does not work, you need to keep your mouth closed and let the user show you how they would interact with the thing you are building.

The last point I want to touch on is something that many of us in the computer industry struggle with. Everyone at one time or another find themselves discussing this topic or doing it. This is over-engineering or building too much for what you need. Joel on software recently posted in relation to a project at Microsoft related to this. The idea here is that you only prototype what you need…keep in mind this directly relates to your audience. In other words, if you are prototyping a proof-of-concept for your boss you may want to have a slightly more polished demo of your work than if you call your buddy down the street and say “he dude, check this out…” Back to the topic, supposing you are supposed to create auto-completion behavior in text boxes on a website, a good prototype would involve creating some text boxes and showing the behavior of the auto-complete. Not, developing the entire page, putting that in a form, and then building the submission controls. Narrow the scope of your prototype. The other reason to do this is because you will only be able to get so much in any testing session. Don’t build the house if you only need to show a door.

Axiom Stack: The Personal Edition, Launched

With the multitude of web frameworks out there, picking a good one can be very tough. Hopefully I can help you pick the Axiom Stack. You wont go wrong with that choice. You can find it at: http://www.axiomstack.com. I would highly recommend checking the site out and reading through their documentation. It will help you understand things that I’m going to skip here. I’m not writing a tutorial, yet, just outlining the features. The Backend/Database Layer

Axiom natively runs on Lucene. What does this mean for you? You have built-in search. Instead of being required to write sql and logic to get resulting data, you pass in the fields you want to search and the term you want to look for. The Axiom guys have also enhanced the Filtering system too. Now, the thing thats keen about this is that they realized that people currently know RDB’s. So, they have the built-in ability to use a relational database straight through the stack assuming there is a JDBC driver for your database. Also, you can utilize the power of both systems at the same time. You can have objects built that are Lucene based, and objects that are RDM based. It’s extremely handy when you have data already stored in other databases that you really would prefer not to migrate (migrating content is a pain…I know first hand).
So, you get some nice data storage services right off the bat.

The Middle Tier/Logic Layer

This layer is by far the most interesting, only because there is so much going on here. You have the ability to write Javascript on the server-side. Thats right I said it, Javascript on the server-side. Fundamentally, all you have to do is create a JS file, dump that into a folder and BAM, you have server-side logic.

The way this works really works is through prototypes. Javascript has a funky way of representing objects. The way it works is through prototypes and the same notion is used here. To create a concrete understanding of the separation you first create a directory in your application. That directory now defines your prototype. The next step is to create a mapping of the properties. This mapping tells the Axiom backend code how to set the datastores up. Then you create a JS file or a few. The JS files are loaded together, so be sure you keep your naming as accurate as possible such that you don’t clobber other functions. If you think about it, if you write a function called getMe inside your prototype definition, and then later do something to the effect of o.getMe = function() {}, you’ve overwritten the original function. So just be careful about naming. I tend to break my functions up by files such that the functions in a JS file all do similar things such as rendering to the screen, macros that happen all the time, etc. The only function you need to be absolutely concerned with is the main method. This is the function that is called by default when you request and object.
It’s extremely easy to create objects in Axiom. You don’t have to do much. Now, if you really want to enhance the functionality you have a great many things you can do. Not only is e4x available (with full xpath support), but all the features of Rhino. This includes list comprehensions, WOOT!! I’ve loved using list comprehensions in python and the fact that I can use them in my Axiom code, even better! So, beyond this there is the whole LiveConnect thing. LiveConnect enables direct access to Java classes and methods through the Javascript. So, if you are finding that the Javascript really isn’t suitable for what you need, or that there is a library out there that already does what you need, you can use the Java code to make it happen. It’s pretty sweet. I’ve dabbled with it and use it for a number of things.

Now, you may have heard of Aptana’s Jaxer server allowing you to write javascript on the server-side while having full DOM capabilities and such forth. But, what you have to do is write html tags that tell the Jaxer server what to run and what not to run. While alright, it doesn’t lend itself to real extensibility IMO. Why do you want to have to riddle your presentation layer with server-side specific tags and code? What happens if you need to change something? Are you forced to change it in both locations? It doesn’t seem practical to me.
With Axiom, the request comes into the server, the server takes the request. If the request is to an object then the main method is used and then processed. If the request is for a method on that object, then that method is processed. This means that you are processing the methods before the templates giving you a lot of flexibility with the kind of data you want to send out. Now, what is a bit tricky here is that the templates, i haven’t gotten to these yet, are considered to be functions which I’ll get into a bit more later. The reason I bring it up here is to point out that you can, should you want name a template such that you can request it from a URL. So, if you have a very basic template or something that doesn’t need the Javascript processing to display it, just name the template what you were going to name the JS function.
The Front-End/Presentation Layer

The frontend is all about TALE. TALE is a proprietary templating language that was developed by the Axiom people. It’s really very simple and that was their intention. There are a handful of directives you can use, and the directives are quite simply just attributes on XHTML elements. You can think of it like XSL without the suck. What I mean by that is that it is not clunky and overbearing. It doesn’t introduce insane amounts of extraneous code into your template just to get logic to work; remember, the logic is done in the JS. The templates allow you to manipulate the XHTML output based on the data you pass into it. A cool element to TALE is that it natively parses JS. So, you can use simple JS to gain access to your information.

To add the last element to your prototype, you create a template inside the prototype directory. This file is now a template for any objects of that type. So in the template you can use proper scoping such that you get access to an objects methods and properties. “this.someFunc()” will look for the someFunc method on the current object. It does that naturally in Javscript, well, you also have that available in the TALE.

My Thoughts

Axiom is a great tool. It provides a ton of functionality and some awesome features that you wont find elsewhere. You’ve got a number of create technologies that have been used with it. And, if you find that you’re missing something, you can write a Java lib or use one to enhance your experience. Really, this does up the ante for other web frameworks out there. It raises the bar. I highly recommend using Axiom as your platform. While the community right now is fairly small, it is active and the people involved are highly knowledgeable about it. Check below for some extra links for good reading.
Some extra links:

• AxiomStack
• AxiomStack: Documentation
• AxiomStack: Community
• Not Rich Yet: Axiom Stack is Go!!
• Mozilla: Rhino

Clearspring and Widgets…What?

So I heard about this company called Clearspring and heard that they were, in a sense, just some host for widgets. At first this made no sense to me, thus I had to check it out to see what it was all about. Here’s my report.

Clearspring has a pretty cool thing going on. I’m not convinced it’s going to be the end-all, be-all method for widgetry, but what I am convinced about is the start of something interesting. Clearspring has taken the idea behind DRY in the Pragmatic Programmer’s guide to development, and are trying to apply that principle to website developers. All you need is a little bit of time, elbow/knuckle grease, and a server that allows for some processing. Hell, if you don’t even need to take in parameters you could just use straight up Apache.

So this is how it works. Developer Nick decides to create an addition to his blog, The Digital Tumbleweed. Lets just say that he wants to add the Library Thing Widget to his blog. Library Thing has, in awesome delight, provided most of the work for us. We just go and select some parameters right? Well, what if I want to use that same widget to do different things on my site. Also, what about tracking purposes? How do I know who else is using this widget? Can I put this widget into facebook too? I mean, really…library thing is all about social networking right?

Clearspring has taken care of a number of these things for us. All I had to do was write a _tiny_ amount of php and then tell clearspring where to look for the code to include. Library Thing takes in the parameters and then generates all the data I’ve requested.

You can see the LibraryThing Widget Homepage, but the widget itself is in action on my site…just peer to the right and down a few scrolls.
Now, I suspect that for some the high level overview is enough to satisfy the craving for info…that or I’ve bored you to tears by now. So feel free to skip this next part…I’m going to get a tiny bit technical. The way this all works is through the use of iframes and flash. No, iframes are not like tiles. Basically whats going on is:

1. You include a bit of JS on your site. Instead of an include or something similar, you are just throwing a script tag in there. Therefore, less processing on your server side. Leave it to the client side…wise or not? Meh…it works for now.
2. This JS is processed client side which makes a call to the clearspring servers.
3. The JS is replaced with new code, an iframe/flash, which shows the results of the processed JS.
4. This new rendering is what you see on your page where you would have normally set your .

The steps and setup I’ve just described are really that simple. The only semi confusing thing here is with the CSS in the widget. If you are including an iframe, your normal stylesheet does not apply to the content in the iframe. Be sure to take care of the CSS inside there.

Is Clearspring the future of web application development? Are reusable, hosted sets of code the next big “advance” in technology? I think only time can tell that, but I will say that they do have a good thing going on. It will be interesting to watch their success.

Undertaking Projects

So I read a post from a friends blog, “Working Small”, and after I read the quote from Linus Torvalds in it I decided it was time to argue points.
Well, the next day at work we talked about the quote because I was curious about perspective on it and it seemed that we both agreed. What we both took the quote to mean was that looking at a project can be a very daunting task. Taking it from start to finish while meeting requirements, fixing bugs, working with new technologies, etc. is quite complicated. However, what we concluded is that the quote really depicts a view on how to go about working on these kinds of projects.

First you must understand that projects for clients are going to need to be thought out, put into a timeline, and then worked on until completion. However, what concerns me is the working on until completion part…thats the portion I do. If you start out by taking some project and work on it’s entire architecture and _then_ begin to develop it, you may have problems.

One of the problems, and possibly the worst, IMO, is the lack of agility. You will have a much more complicated time restructuring a system if you go through the architecture in advance of any of the development. Instead, if you focus on smaller tasks you will have a better idea of the start and end points and be able to develop with a clear vision in mind. Also, you have more ability the change an API or develop new ones as you are developing it due to certain needs. You are less likely to be able to change due to clients needs. Many projects go for some period of time and due to this, requirements may change. The client may decide over that period of time that the functionality of requirement ‘x’ actually needs to do ‘y’. In a hard fast architecture you have 0 agility. Instead, you need to add more to the application which creates bloat and functionality that may not be used. This leads to the inability to refactor your code effectively without leaving broken windows.

Now, these ideas are not new. The guys over at the Pragmatic Programmer have been talking about it for some time, however, I thought that the quote was the common link between what those guys are saying. It really embodies the view most developers should have when undertaking a project. While some architecture is important, it should not be the entire focus of the initial stages of development. There is something to be said about building prototypes and hashing ideas out before really designing the architecture of the final product.

You must have some idea of where you want to go, but that does not mean you should throw agility and prototypes out of the window.