Scratching An Itch

A couple friends an I were talking a few weeks ago about building applications. There are a handful of reasons for creating websites or web apps. The topic of discussion that day was about applications that do something you need and not just about the ideas, but about finishing those projects.

To say that creating applications to scratch your own itch is fun is an understatement. Usually you are heavily invested in the idea and so you have a lot of drive to start it out. I love the idea stage of ideas. I have a lot of them and have a whiteboard in my apartment covered with ideas and mini-architectures around those ideas. Yet, most of them don’t seem to happen.

I know that I am pretty well motivated to create stuff that I’ve bought into, so what are the reasons that these projects seem to stay stagnant in the idea phase.

Exhaustion. The first killer is exhaustion. The reason this ruins the drive to do more is that when I get home from work where I’m trying to solve other problems all day, I’m not necessarily focused on pushing through mine. Doing more work on top of the work I’ve already done doesn’t sound like loads of fun to me.

Ideas are fun. Plain and simple, work is hard. Running around thinking “I can make it fly and produce magic ponies too!” is great. But, when I’m the one that has to produce the result of the idea phase, it becomes a major stumbling point to getting something done.

Broad ideas are broad. Without defining a limited scope to what you ought to be doing, your focus and ADD will take you in 90 different directions. Rather than getting something done, you’ll be focused on starting a number of different things and leaving others to hang out in a partially working state.

Lacking skillset. When you are lacking a certain something to get the project finished, you will find it very frustrating to not be able to just hammer on the rest of your idea.

So what do you do?

Focus. Harness that N.A.D.D. and get something created. The original idea doesn’t need to be your “pony, dragon, rocketship” release. It’s your proof of concept (POC). You need to see that your idea _can_ work. You being the brainchild behind it, make it work and then add onto it.

Give yourself a deadline. When I want to really get one of those projects on my whiteboard done, I give it a deadline for a POC. This means that if the idea isn’t done by that date that my interest either isn’t there, or that the problem is far more complex than I originally thought. If the idea is too complex, I’ll know early on and will have to rethink my approach. Otherwise, I should be able to push through. And, generally by applying a date to it I know that my interest level is there. It’s just a matter of doing it.

Have people to push you through the “hard times”, as my friend called them. The hard times are when it’s not easy going, when you need to sit and actually think about the solution and the best way to implement it. Often times when the hard problems hit there is no obvious solution. This makes it quite a bit more complicated and if you can have someone around to get you through those points, you’ll be more likely to succeed. The point here is that you have someone or a group of someones to hold you accountable for moving the project forward to completion. Believe me- this doesn’t always work, but when it does it is a good way to keep yourself on track. Also, it will allow you to get feedback about what you are doing. To steal an idea from the agile methodology, “Release early and often”. Meaning, put your project out there sooner rather than later and make smaller incremental changes. There are _many_ benefits to this practice, but those are better left for another post at some point.

Lastly, if you don’t have a skillset, you are fine. Just keep in mind that you might not be able to get some part of the project working for a bit. Either read up on it and learn it or outsource it. Phone a friend, check in the freelancing sites, etc. I know that personally I am _horrible_ with design. If you want squares with black and white, I’m your guy. The point here is knowing your limitations, and finding a way to push through them. This tends to be my biggest competitor to actually producing a full project for public consumption.

Now, if I were in your shoes I’d be thinking, “What do you know? You’ve got a list of untouched projects.”

Although that is true, this is what I’m using to start doing to tackle that list of projects. I’m curious to know how some of you hunker down and get your stuff done. I know that some people use Backpack and/or Basecamp. I know others use events like Hackday to get their ideas done.

I’ve tried those and they didn’t work well enough for me. The Hackday approach is what has worked best for me but there are tools that are other tools helping me out here.

The first tool helping me is Git. This version management tool allows for some very nice things. If you haven’t taken a look at Git yet, do so. You’ll be glad you did. Another tool, based on Git, is Github. While Git itself is great, having a central repository is very nice for having your source be always available. I’ve been putting all of my projects up there and while the decentralized model of DVCS is still alive and well I know that I will always have access to the source so long as the internet exists. Lastly, telling people about the projects is the key to me getting the work out there. I have people to push me through those hard times and friends that can help me to complete the design work that I seem to be exceptional with.

Back to that conversation I was having with my friend; one thing I realized in that conversation is that usually when you have smaller projects or ideas they do not require a team of people architecting a larger solution and in fact that collaboration can hinder your progress. They require a few hours of attention to just focus and hammer down on. When I started doing that, a few of my projects started to take shape. You just have to dive in and see where you come out. You don’t need the communication overhead produced from more people on the project.

Ultimately, I don’t know that this approach is going to work in my attempt to get some projects done, but so far so good. I’ve thrown a bunch on Github and am going to be displaying them here as well. I’ll be looking forward to some feedback. The licensing for now on most of my stuff is AGPL, but that could change shortly. The reasons for this are all the topic of another blog post about licensing.

Hire the Meta Thinker

In the past few months I’ve seen suggestions such as fire the workaholics all the way to fire the underachievers. Then you’ll see suggestions about how to hire and what to hire. What I don’t understand is that everyone is looking for specific traits, “has xx years working with J2EE, has handled a project from start to finish, has lead a team of 5+ people, has a 4 year CS degree”. What is the point? None of the above is going to really help when it comes down to producing something good. I say skip the crap and focus on thoughts and thought process. Hire the meta thinker.

I find that the quality produced from meta thinkers is far superior to the contrary. The reason I say this is because the person writing the code cares about asking what and why. Simply asking “What is this method supposed to accomplish?” and “Why am I writing this method?” starts the thought process into what will inevitably be a much more powerful set of tools.

Not only will those tools, functions, libraries, etc. be more powerful, they will be reusable. The reason I say this is because a meta thinker will think about the practicality of the functions or libraries. Meta thinkers aren’t concerned with the number of lines being produced, in fact, it tends to be the reverse. A meta thinker strives to drive the number of lines required down to the optimal level. The optimal level being the most efficient level. This ties into the “what”; the reason I am writing this function is because it allows me to create an object within the database. A meta thinker asks, how can I make this universal. How can I make this work outside of my specific scenario.

The other reason a meta thinker will produce better code is because of the “why”. A meta thinker understands that not everything requires separation, but attempts to see whether it makes sense to do so. To develop great software you will want some people who ask “why am I writing this?” One thing that will come out of that question is the finding of libraries or other tools. Also, you’ll find that the task at hand may be too complex for the purpose and realize and architectural fault in the system…Why am I really required to write ASM for this web app? Is the speed increase necessary? Maybe it’s time for a new framework. A meta thinker will find flaws less obvious than this one.

Beyond this, a meta thinker is someone you want around because you can bounce ideas off of them. A meta thinker will be able to help you flesh out ideas. These people will be able to expose issues or bless your concept. You are nearly always guarenteed to have a good idea of the more difficult parts of the operation once you’ve talked with these people.

Another reason you want to hire meta thinkers is because they are interesting to talk with. Not only do you build your level of understanding, you start to think differently. You learn and adapt your thought patterns similarly such that you’ll find yourself asking why a lot more. You’ll find that, after a period of time, you are the meta thinker.

This process is enlightening. You find that you try to solve a problem before solving the problem. Instead of diving into an issue you will think about it before and then attempt to solve the problem. This produces cleaner code and better performance, both in code efficiency and in the rate at which you write your code. By meta thinking, you find that you are solving the problem of how to solve problems. You are finding out how to write reusable methods by thinking about how to write methods. Your thoughts begin to change and your notion of just getting it done changes because you know that there is a better way.
Now think about it, would you rather work with someone that comes into the office and just does what is necessary for the day and then heads home, or would you rather work with someone that meta thinks about problems and come up with a crazy solution to a problem? I prefer the latter, it is just more interesting to me but, bottom line, I would prefer to gain more value for my buck.

A Day in the Life of an Open Source Project

Open source projects can be a challenge to manage. Why? Imagine this, you have a product you hold near and dear to your heart. You’ve poured blood, sweat, and tears into this thing and you have finally hit something you think is worthy. You ship it out to the world and say, “Here is my code: feed, digest, and enjoy.” Twp hours later you are bombarded with 40 emails telling you how horrible an idea using algorithm X, Y, Z was and how you should see that obviously W, X, Y is the preferred method. Then you see others that make note of your lacking skills in coding and that you should quit your day job. Next you see about 5 patches that need to be reviewed to fix bugs in your system. And lastly, you have requests to help enhance your product. This makes you want to pull your hair out- assuming you have any left at this point.
The picture I’ve tried to paint, I’m not an artist so forgive me, is not a very shiny or happy one. In fact, that is one of the things that turns people off from open sourcing products. Sure there are other reasons, but managing open source projects is very hard work. So, how do you make it easier. You should note that the community is not going to letup at all on the product providers. Everyone has an opinion and thinks it needs to be heard/read. So, without getting into the why’s or the benefits of open source, lets discuss how.

What things can be done to better control the flow of an open source project.

1. Patch Management- Bug fixes are going to come in, but to control those you could set up a bug management system. There are a number of bug management tools available; you just need to find them and use one. This will allow you to continue working on the things you need so that you can schedule 30 minutes at the end of a day or every other day to review any patches that came in and either include them or send them back. Not only will a bug system allow you to manage patches, it will provide a clear roadmap for your users and other developers. Thus, people wont step on each others toes. Two bug management systems that come to mind are Jira and Trac. Atlassian has a deal for open source projects as well, so take a look.
2. Enhancement Management- Branches. Branches. Branches. I’m not talking about the crap-tastic ones that SVN provides. I’m talking about real branches; DVCS branches. Go out and experiment a bit with Git, Bazaar-NG, and Mercurial. See which one tickles your fancy and pick it. You will have a far better time running your project with one of those rather than the alternative. And, if you don’t want to deal with hosting one of those you can use services such as Launchpad. What having one of these allows for you to do is create a branch that only the ehancers can push changes into, This effectively keeps the sources separate. But, when time comes to review the enhancements and push them back into the main branch, you can do this with a decent amount of ease.
3. Peer Review- You will never be able to break the flood of “you suck” commentary. However, you may be able to channel this behavior into positive, constructive criticism. Peer code reviews are great methods to keep you on your toes and make sure that you are always learning. My thoughts are that there is always someone more intelligent than you are out there. So listen. You can use some tools that provide a great way for people to write back on the code you are producing such that the comments go from “Why in the world would you use X?” to “Algorithm X is inefficient here because you loop over the data set twice. Use Y instead.” The commentary will become productive and useful. Tools do exist to aid in review; take a look at Crucible.
4. Ease of Contribution- Remember open source is about the community. You can’t half ass this. People have to be able to contribute in one way or another. And, they will know if you are just giving them the run-around. Therefore you need to allow for people to easily contribute. Many times people will be able to squeak 30 minutes in here or 2 hours there. And, if they have decided to spend that time on your project, you should be grateful. They could be doing something else. Thus, if you make it easy for them to help you, you’re likely to see better results from your open source-ness.

If you keep these tools in your toolbox for helping you to run an open source project then you are more likely to succeed with your project. Keeping the right attitude will also help. Be sure to take all criticism as good criticism. It can be hard since your code is generally close to your heart, but nobody is perfect and having a good attitude towards criticism will always be a benefit to yourself, whether in an open source project or not.

What approaches do you take when working on open source projects? How do you contribute? What are the steps you use to push your project to success?

The Next Holy War: Distributed VCS versus Centralized VCS

I often find myself being an advocate for something specific in the technology field. And, many times this relates to the common “holy wars” that most of us are familiar with. I sit on the emacs side of one, and the open source side of another. You can immediately understand what I’m referring to when I say “emacs”. The next topic for war I believe is going to come down to version control. If you want more proof, take a look at Derek Slager’s DVCS Myths post.
Currently there are two main types of version control and source management, centralized and distributed. I’ve used centralized systems for quite some time and although they have served a purpose…I find them to be outdated. Now, hear me out before getting frantic. The software industry is changing. We are in the midst of evolving the way we work. We want to work bigger, faster, better, stronger, right? There has been a push over the last 10-15 years to revamp our development methods. What I’m referring to here is the waterfall to agile approach of things.

The waterfall approach is about building very hard speced systems over the course of “x” to completion. Once the system is complete, you test, debug, etc. This is often very clunky, doesn’t describe the entire system, and takes an extremely long time to obtain any tangible assets. By this I mean that nobody can see a working system part of the way through. Thus, expectations are usually high leading let-downs as a result. The agile approach, in many ways, was created as a way to attack the problems at the core of the waterfall approach. Also, certain agile approaches can help do more such as improved development efficiency. With this in mind, and understanding that I sit on the agile side of that fence lets talk about source code management (SCM).

Centralized version control was fantastic. It provided the foundation for storing code in one place such that developers could work on code and then push it back into that one location. Not only was this able to happen, but we could lock, merge and tag. These are all good things by the way. What good would any SCM be without them. The lacking areas are the repository and the workflow.

Since we talked a bit about workflow above as it pertains to development, lets talk about how it relates to source management. In the likeness of the Pragmatic Programmers we know a few things to be good. That we want to refactor early and often. And, we do not ever want to settle with broken windows. These few things being said, lets discuss them as they pertain to workflow. In any SCM we can accomplish each, right? We checkout code, modify it, and commit new code. Centralized or decentralized. But, I’ve found it easier to manage the above “good things” within distributed version control system (DVCS). When I am working on refactoring code, I want to have an experimental branch. This allows me to create features, change API, and muck with the general internals of a codebase without destroying the greater good. That brings me to broken windows. If all developers commit to a central repository, then all code is based on said repository. Thus, if you have a team of n (including yourself), you can stop n-1 people from working for h number of hours in a day by commiting some code that you thought worked.

With a DVCS, you are commiting your code to a branch. This means that you are not commiting to the central repository. Sure you have to deliver your changes at some point, but you are more likely to have your code in working order. Also, it allows for easier testing of your code, since your code will be a subset of all new features and changes. So, when we refactor our code or write new features we are setting ourselves up better by having the DVCS.

Now that I’ve touched a bit on the workflow we can talk about the repository. Distributed version control systems are just that, distributed. The point is to make it easy for developers to make changes while being anywhere. The way this works is with distributed repositories. Distributed systems work by branching. Although the notion of branching is built into SVN and other centralized systems, they don’t work the same.

Distributed systems, as seen in the image, mean that everyone is a node, or using our terminology, a repository. Rather than just feeding off one main location, I can pull from my friends repository, push code changes up there or to a different repository. Branching allows for this sort of behavior.

So, when I modify code on my system and then commit it to my local repository, I can push those changes along to whomever or they can pull them down from me, and everyone has my latest changes.

So, I believe the thought going through your mind at this point is, wait…You’re contradicting yourself. If I want to commit often and refactor even more often then I’ll likely break something. Thats not good. The point here is that you shouldn’t work directly in your local repository. You should branch the repository and do work within the branch. For instance, say you are working on an OS. You can have a branch for networking. I can then grab that branch but if I have to implement some experimental IPV6 functionality, I’m not going to do so such that everyone gets those changes. I’ll branch the code base and do it there rather than expose that code to everyone. With a centralized system, you can branch the code…but merging the modifications back would be complicated and cumbersome, and in some cases not possible. If you need more proof take a look at Griddle Noise: What a DVCS gives me.
This being said, the industry uses centralized version control for most work currently. I feel that the change to agile programming should be accompanied by a SCM system that easily supports agile processes. A DVCS goes hand in hand with a distributed source management system.

There, I’ve stated my thoughts on the point. Flame away.

Prototyping in a Digital Age

I attended my first RefreshDC event this past Thursday. Pretty interesting I must say. For those unaware of the procedure; the venue is a room with projectors, seats, and a podium. Dude stands at the front and talks while taking questions. Clean, simple, and if kept to the timeframe…engaging.

Todd Warfel was the man presenting and his topic was prototyping. I’m a fan. I have to say he had some really interesting suggestions and ways to grab feedback. From using paper and colored cellophane to a miniature layout in html, he had a plethora of ideas. In my opinion that was where the most value came. His presentation talked of seven steps to take when prototyping for someone which were very generic and sounded like marketing babble, but when he started to talk between the points he provided a great deal of information.

The first think I want to make note of, because I feel it holds a great deal of relevance to what I do, is when to prototype. Todd mentioned that you should prototype early and often…I believe that he has this right. Early prototypes, especially when you are trying to gain requirements, is pivotal in a good product. You need to put something in front of people and find out what they like and dislike about something you plan to put time, sweat, and money into. Doing it any other way just doesn’t seem to make a lot of sense to me. Why build the requirements for a system when you haven’t thrown it at a user yet? You’ll spend significantly more to fix it later on in both branding and development work. Also, prototyping often allow you to focus your effort. You can parallelize the work of the next feature with the development of the last. And as with anything, the more you do it the better you get.

The next thing that I felt was a great takeaway was his mention of bringing in a good mediator. I don’t believe this was a point in his presentation but he mentioned it between points. In essence he mentioned that a mediator for user testing and prototyping should not try to help or suggest the answers to people. It’s generally very difficult to not help users when they are struggling, especially when you built the thing they are trying to use. However, if you really want good feedback on what does and does not work, you need to keep your mouth closed and let the user show you how they would interact with the thing you are building.

The last point I want to touch on is something that many of us in the computer industry struggle with. Everyone at one time or another find themselves discussing this topic or doing it. This is over-engineering or building too much for what you need. Joel on software recently posted in relation to a project at Microsoft related to this. The idea here is that you only prototype what you need…keep in mind this directly relates to your audience. In other words, if you are prototyping a proof-of-concept for your boss you may want to have a slightly more polished demo of your work than if you call your buddy down the street and say “he dude, check this out…” Back to the topic, supposing you are supposed to create auto-completion behavior in text boxes on a website, a good prototype would involve creating some text boxes and showing the behavior of the auto-complete. Not, developing the entire page, putting that in a form, and then building the submission controls. Narrow the scope of your prototype. The other reason to do this is because you will only be able to get so much in any testing session. Don’t build the house if you only need to show a door.