Digital Tumbleweed

Internet Censorship: SOPA and PIPA

Wed, 18 Jan 2012 07:21:00 -0800

While posterous will not allow me to add JS to blackout my site, I figured I'd use this space to throw up some information for those unaware of these two bills.

Please, if there is one thing you do today. Educate yourself on SOPA and PIPA. It is extremely important.

https://www.google.com/landing/takeaction/sopa-pipa/

http://www.blackoutsopa.org/

http://sopastrike.com/

http://en.wikipedia.org/wiki/Main_Page

http://theoatmeal.com/

http://fightforthefuture.org/pipa/

PROTECT IP / SOPA Breaks The Internet from Fight for the Future on Vimeo.

If you do something else, get in touch with your congress people.

Permalink | Leave a comment »

NodeJS project management with NPM

Fri, 19 Aug 2011 05:06:00 -0700

I've seen questions for this topic come up repeatedly so I figured I'd throw something down to refer people to. :)

If you're interested in NodeJS, and I'm betting that you are, then you'll be wanting to make a project. There area few things you can do. Many, not all, people in the NodeJS community use Git and Github for source control and hosting respectively. You'll do fine hunting around there for things people have made on Github. Alternatively, the NPM repository hosts stable versions of each module and provides some conveniences for installation.

At the start you may be tempted to just something like:

npm install module

I'd recommend against this. While it will work in a pinch. But, a better approach would be to use the package.json file. At first it seems strange to do so, but it becomes natural very quickly. With this approach you get the benefit of having a file you can check in. This means that you can throw it in your versioning system and once done, type:

npm install .

This will install your current project and all of it's dependencies. It makes it really nice when moving around machines. I usually work on two machines and then deploy on another. Having a file that makes installation easier is really nice. The file itself would looks something similar to this:

{
  "name": "packagename",
  "description": "A description of your project...even though you wont necessarily be putting it in NPM.",
  "version": "0.0.1",
  "author": "Your Name Here (http://github.com/user)",
  "engines": { "node": ">= 0.4.11" },
  "repository": {
    "type": "git",
    "url": "http://github.com/user/project.git"
  },
  "licenses": [{ "type": "MIT" }],
  "bugs": {
    "web" : "http://github.com/user/project/issues"
  },
  "dependencies": {
    "express": "*",
    "bcrypt": ">=0.2.3"
  },
  "devDependencies": {
    "nodeunit": ">=0.5.1"
  }
}

Note that you do not need to put your project in NPM. We're talking about managing your dependencies and other project information. However, one caveat is that this file needs to be proper JSON.

If you're interested in more, take a look by typing:

man npm-json

Hope that helps. Happy Hacking.

Permalink | Leave a comment »

So you want to learn NodeJS and see what the fuss is about?

Sun, 30 Jan 2011 10:18:00 -0800

If you're looking for a coding tutorial you're in the wrong place, but if you'd actually like to learn NodeJS then stick around. This will cover the basics so that you can understand why things work the way they do. A base understanding is what you should start with. So, lets get into it shall we?

A Brief History - SSJS in the Making

Over a few recent years a handful of projects has tried to bring Javascript to the world of serverside programming: RingoJS, Axiom Stack (which I was involved with), Jaxer, and NodeJS among others. There are also a few javascript virtual machines out there: Rhino, V8, and Spidermonkey. This abundance of tools has allowed for those who really love Javascript to dive head first into server side development.

The project that has gotten the most attention for bringing SSJS is NodeJS. The attention it has received is largely due to two things: the way with which you write Javascript and the speed at which you can do so. You see, NodeJS set out to work through events and callbacks. What this means is that your traditional concepts around procedural programming were just thrown out the window. And, while functional programming has been around for some time, this is a wonderful blend of functional and evented development. There are arguments for and against this asynchronous approach, but I'm not interested in discussing that here. The point is to get you understanding it so that you can form your own opinions. So I'll explain why this works the way it does.

A Look at the Internals - The Event Loop

Events and callbacks are used because the core of node is an event loop. This means that you have a loop processing all of your code. If your code is procedural, step-wait-step-wait and so on, then you will eventually take up the entire event loop with your code and nothing else will be able to be processed. This would normally be fine, but most applications are complex that require some database interaction or writing to files. Additionally, you may need to send data across a network. This is where things get a little hairy. When you block the event loop you stop other code from executing and thus waste CPU cycles. The traditional example with node is this: if you make a database call then you have to block until you get a response from that database. This means that you cannot do anything else until that initial call has returned. This is known as blocking and will halt the event loop until it has finished.

var db = new DB(); var rows = db.getAllRows(); console.log(rows.length);

But, imagine this. You create a function that would take the result of your database call. Then you pass that function to your database call such that when the result set is returned, your function is fired off.

var db = new DB(); db.getAllRows(function(rows) { console.log(rows.length); });

I'm affraid that this alone is not enough to keep the event loop uncloged. There are two things to note. The first is that the db library you're using needs to be using events. And the second thing is that you can still make enough calls that you reach the system limits for your event loop and clog it, but in most cases this is either bad design or a good problem that you solve with more hardware.

Evented Libraries

Libraries don't event themselves. The developers need to be cognisant of that approach when creating the library. For node this can be in either C/C++ land or in JS land. Either way the approach works using the notion of events. For instance, lets say that I have a database library. In this library I make a request for all the rows in the db using the connection. This could take some time so I want to make sure I'm not blocking other execution and thus take a callback as the parameter. In the getAllRows api I set my code up such that when the result returns from the database itself your callback is fired with the response from the server.

function getAllRows(cb) {
conn.on('results', cb);
conn.getRows(0, conn.length);
}

Although this is obviously a contrived example, you can see that what I've done is used an api such that when the results event happens, we fire the callback. Then, we make a call to the conn, db connection, object to get rows 0 - the size of the rowset. Although this isn't the most powerful example, it shows the essentials of an evented architecture. We just hang onto the callback in memory allowing the event loop to continue on with it's other tasks until we get a response from the database call, which is when we make our move. I've stated this point about 5 times by now I'm sure, but I hope it's really clear. This approach is asynchronous. That's 6. :)

How Does That Connection Become Asynchronous

If you look, there is an implicit assumption in the code above, that assumption is that conn itself is asynchronous. That works for my example, but it doesn't really explain anything. So, how does the connection become asynchronous? This comes back to node.

In node, we have something called an EventEmitter (refer to the link for the API). This is the backbone of eventing in node. The EventEmitter is a module whose backbone is libeio. This library provides the functionality needed to make your code asynchronous in NodeJS. The EventEmitter as a module provides you with publish and subscribe features so that you can make any code you write use events. So, the connection module would be required to use an EventEmitter in order to provide that asynchronous functionality.

A simple event emitter might look something like this.

var sys = require('sys')
,events = require('events')

function Eventer() {
    events.EventEmitter.call(this);
}

sys.inherits(Eventer, events.EventEmitter);

var eventer = new Eventer();

eventer.on('new', function() {
	     sys.debug('test');
	 });

eventer.emit('new');

Setting NodeJS Apart

What sets node apart from the competition is that there wasn't a set of libraries pre-built using other design paradigms. So, people have built a massive collection of libraries from the ground up to give you your chance at embracing the power within node. Hopefully by now you realize that you can do some awesome things using NodeJS. Armed with your new understanding here are a few links to get you started on codifying it.

NodeJS Site
How To Node
Node Wiki
NodeConf
NPM - A NodeJS Package Manager for Libraries and other Modules

Permalink | Leave a comment »

Locales and LC_CTYPE

Sun, 25 Apr 2010 21:14:00 -0700

Recently I added another slice to my list of servers but whenever I add slices on Slicehost with Ubuntu, they come with the locale problem. You might recognize it. The following shows up in almost ever source compile you do (frustration mounts):

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_CTYPE = "en_US.UTF-8", LANG = "en_US.UTF-8" are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C"). locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory

Then, after reading the error you see that it doesn't even have en_US installed! *GRAH* Well, there is a simple solution. In my case I use en_US, but if you use something different then make sure you specify it.

cd /usr/share/locales
sudo ./install-language-pack en_US
sudo dpkg-reconfigure locales

This should fix the problem. To see what your system is set to use, you can do the following: locale

It will spit out something like this:

LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC="en_US.UTF-8" 
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8" 
LC_PAPER="en_US.UTF-8" 
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Hope it helps.

Permalink | Leave a comment »

Karmic and Phonon: Not Found

Thu, 18 Feb 2010 13:24:00 -0800

This AM I was attempting to update my existing version of Choqok and received an error. I thought I'd share since it did take a couple of minutes to find a solution for fixing. Dependencies:

I'm running Kubuntu Karmic, that's the KDE Ubuntu 9.10
I pulled the latest Choqok on Gitorious which is where I hit my issue.

The concepts will likely work elsewhere, but not within the nice packages provided in aptitude. The Problem

~/Code/choqok/build$ cmake -DCMAKE_INSTALL_PREFIX=`kde4-config --prefix` ..
-- Found Qt-Version 4.5.2 (using /usr/bin/qmake)
-- Found X11: /usr/lib/libX11.so 
-- Phonon includes NOT found! 
CMake Error at /usr/share/kde4/apps/cmake/modules/FindPhonon.cmake:63 (message): 
Phonon library or includes NOT found! Call Stack (most recent call first): /usr/share/kde4/apps/cmake/modules/FindKDE4Internal.cmake:559 (find_package) 
/usr/share/cmake-2.6/Modules/FindKDE4.cmake:81 (FIND_PACKAGE) CMakeLists.txt:3 (find_package) 
-- Configuring incomplete, errors occurred!

Short Fix Use aptitude

~/Code/choqok/build$ sudo aptitude reinstall libqt4-phonon-dev
Reading package lists... 
Done Building dependency tree Reading state information... 
Done Reading extended state information Initializing package states... 
Done Writing extended state information... 
Done The following packages will be REINSTALLED: libqt4-phonon-dev 0 packages upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded. Need to get 0B/57.7kB of archives. After unpacking 3,441kB will be freed. Do you want to continue? [Y/n/?] y 
Writing extended state information... 
Done (Reading database ... 
264392 files and directories currently installed.) Processing triggers for man-db ... 
Processing triggers for desktop-file-utils ... 
Processing triggers for hicolor-icon-theme ... 
(Reading database ... 
264183 files and directories currently installed.) Preparing to replace libqt4-phonon-dev 4.5.3really4.5.2-0ubuntu1 (using .../libqt4-phonon-dev_4.5.3really4.5.2-0ubuntu1_i386.deb) ... 
Unpacking replacement libqt4-phonon-dev ... 
Setting up libqt4-phonon-dev (4.5.3really4.5.2-0ubuntu1) ...
Reading package lists... 
Done Building dependency tree Reading state information...
Done Reading extended state information Initializing package states... 
Done Writing extended state information... 
Done

Long Fix Use apt-get

~/Code/choqok/build$ sudo apt-get remove libqt4-phonon-dev
Reading package lists... 
Done Building dependency tree Reading state information... 
Done The following packages will be REMOVED: kdelibs5-dev libphonon-dev libqt4-phonon-dev 0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded. After this operation, 13.6MB disk space will be freed. Do you want to continue [Y/n]? y 
(Reading database ... 264392 files and directories currently installed.) 
Removing kdelibs5-dev ... 
Removing libphonon-dev ... 
Removing libqt4-phonon-dev ... 
Processing triggers for man-db ...

Now, reinstall the libraries that were uninstalled.

~/Code/choqok/build$ sudo apt-get install kdelibs5-dev libphonon-dev libqt4-phonon-dev
Reading package lists... 
Done Building dependency tree Reading state information... 
Done The following NEW packages will be installed: kdelibs5-dev libphonon-dev libqt4-phonon-dev 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 0B/1,229kB of archives. After this operation, 13.6MB of additional disk space will be used. Selecting previously deselected package libqt4-phonon-dev. (Reading database ... 
262408 files and directories currently installed.) 
Unpacking libqt4-phonon-dev (from .../libqt4-phonon-dev_4.5.3really4.5.2-0ubuntu1_i386.deb) ... 
Selecting previously deselected package kdelibs5-dev. Unpacking kdelibs5-dev (from .../kdelibs5-dev_4%3a4.3.2-0ubuntu7.2_i386.deb) ... 
Selecting previously deselected package libphonon-dev. Unpacking libphonon-dev (from .../libphonon-dev_4%3a4.3.1-4ubuntu1_i386.deb) ... 
Processing triggers for man-db ... 
Setting up libqt4-phonon-dev (4.5.3really4.5.2-0ubuntu1) ... 
Setting up kdelibs5-dev (4:4.3.2-0ubuntu7.2) ... 
Setting up libphonon-dev (4:4.3.1-4ubuntu1) ...

Fin

The basic idea here was to reinstall the libqt4-phonon-dev library. This made it so that the include files were back in place and I could compile source code. Now, you should be able to continue compiling your phonon based source. In this case, as stated above, I was compiling Choqok. But, I doubt this issue is relegated solely to Choqok compilations. I just haven't yet had the opportunity to compile other KDE4 based apps on Karmic Kubuntu.

Permalink | Leave a comment »

Buzz Words Killed the Radio Star

Thu, 28 Jan 2010 03:04:45 -0800

I'm ideating a method proposition to add enterprise grade value which will synergize your verticals. *cringe* Hold on a second...I need to go wash off the dirty. In all of what I typed above, I didn't actually tell you _anything_. But, what you expect now is some thing that should make your things work well together. Why don't we just say that? Because people expect exact answers and the above sounds more like an exact answer! Keep in mind that most places wont do anything nearly as drastic as what I posted above. Where To Begin?

Buzz words suck. Often times these words start as a way to differentiate products and/or services. While grande, buzz words tend to lose their meaning when _everything_ and/or _everyone_ uses them. When people adopt them on a large scale you're not actually saying anything. Most people can get the gist of what you're attempt to say because most people have a brain that works reasonably well, at the very least better than this weasel. However, you lose credibility when you start throwing those kinds of words around. I've been, for a while now, trying to figure out why they are so popular. I'm not sure where it started. The main reasons for using them, that I can think of, are differentiation and filler. Differentiation The attempt to be different. This works out reasonably well when you're the first person/company to say that you are an "enterprise" solution. But, when every competitor uses the term, SURPRISE, you're no longer different. Or are you? What about the feature you can create a pony out of thin air? I'd say that likely differentiates you, wouldn't you? Stick to the features and to your capabilities. Those should be what differentiates you.

Filler If you find yourself with a lack of descriptors then use the ones you can think of. When you "innovate" in the market, what are you really doing? People thirst for information so give it to them. You don't need to ramble on, but providing some detail is a great teaser. The movie industry uses teasers to pique your interest. You should do the same. Additionally, using buzz words doesn't make you sound any more intelligent. In fact, it often does the opposite because you aren't providing substance. More Proof If you need more proof that buzz words are everywhere and how irritating they can be. Sit at your next "all hands" type of meeting. Listen to the specific words being used. You'll notice that a lot of these phrases aren't really telling you much. It's almost like eating at McDonald's, you might get something that resembles a message, but really all your getting is an ill feeling in your stomach. Try playing games like buzz word bingo. Another that I've played is to come up with a list of words and assign points to those words. Each person picks a word they think will provide the most points. Give each person a word to tally and then collect the tallies at the end. Multiply the tallies for the word by it's value and those are the total points. Once you have the totals you send them out. The prizes are your choice, I tend to prefer a free lunch, but it at least makes the 1-2 hour snore fest more exciting. The point is that you'll find out just how many buzz words and phrases are used to try and pitch you information and direction. Where To Go From Here Well, I'm certain I've used buzz words in the past. With so many floating around, it's bound to happen. Fret not, you can stop using them and gain back your credibility. Make a conscious effort to stop using them and give the terms meaning again. Also, you'll be generating more intelligent conversation. This, to me, seems like a positive. Suggestions You may be able to get a clearer picture of exactly what I'm talking about with some examples. So, here are some off the top of my head.

Enterprise - We can handle a lot of [users|documents|]. When you say this you'll need some statistics. So have them ready.
Web 2.0 - Our web application is primarily built using Javascript, AJAX, and some aesthetically please designs. We are also standards compliant.
Synergy - Our product works well with . Or, this will allow to work well with .
Best of breed - We've used technology. This gives us the ability to do .
Leaders in - Hmm... No you aren't. Just accept it now and get over this phrase. KTHXBI.

Authenticity One thing I touched on above was authenticity. When you stop using gibberish in your messages you gain credibility. People start to listen to your messages because they are gaining something from it. If you are interested in some ways to approach this take a look at Personality Not Included. I read this some time ago and really liked much of what Rohit had to say. Go Forth Go now and conquer! Take out those editors, rewrite those presentations, press releases and websites. Get the word out that you wont put up with buzz words anymore. And, if someone tries to push you into it, you can always threaten them.

Permalink | Leave a comment »

Apache Modules, SEOversite and Debugging C

Thu, 19 Nov 2009 15:30:03 -0800

At Axiom Software we're working on the next release of SEOversite. It's been exciting. This is basically a tutorial on debugging the Apache HTTPd side of things (keep in mind that this is as much an informational thing for you as it is a reminder for me. I assume that I'm going to have to re-read this at some point). Background In production, we've been using Apache on Ubuntu Server. This has worked out well seeing as aptitude rocks pretty hard. But, we run into some issues doing debugging and profiling. It's easier to have the code all within one place, so for development purposes we're pulling the source so that we can specify the configuration and compiler options for Apache. Debugging

What other tool do you use to debug C code than the GDB? With any language you need a way to be able to properly debug your code and usually print statements just don't get it done. So, why stop at the ability to print out from the code when you can actually step into your functions, print out variables, and see back traces. With normal C applications you can simply just load the binaries into GDB: # gdb This doesn't quite work in our setup though. We are in the situation where we need to debug modules, not just Apache. And, to really be able to step into the code we need to compile it appropriately. When we're dealing with setting up our module for debugging we need to make sure we pass in the -g flag. The -g flag just makes it so that your code can be debugged with gdb. So, we have our module which we'll call seoversite.c. We're looking to debug so our build looks something like this: ~/seoversite/# /opt/apache/bin/apxs -ci -Wc,-g seoversite.c This turns on debugging within the binary which means that we are letting GDB process the source of our file. With apxs, the -Wc, flag allows you to pass compiler flags along for when your code is actually built. We then have a really simple startup for GDB. /opt/apache/# gdb bin/httpd As soon as this fires we get a prompt that lets us specify what we want to do. Today I was debugging a simple method that processes a URL so I'll go through some of that. When we start up gdb we're presented with a prompt. This is where we need to enter some input.

(gdb) b get_fullurl
Function "get_fullurl" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (get_fullurl) pending.

Because our module is shared, gdb doesn't find the method signatures right away. That's ok. We'll wait till they load on the request. Basically, the above puts a breakpoint on the method call get_fullurl. Thus, any code paths that use get_fullurl will halt activity until processed in gdb. So, we start the server.

(gdb) run -X
Starting program: /opt/apache/bin/httpd -X
[Thread debugging using libthread_db enabled]
[New Thread 0x7fce2930d760 (LWP 14174)]
warning: Temporarily disabling breakpoints for unloaded shared library "/opt/apache/modules/seoversite.so"
[Switching to Thread 0x7fce2930d760 (LWP 14174)]

The command we typed was "run -X" meaning, start the binary with these command line options. -X is the debugging flag for Apache. So, we're telling it to start Apache HTTPd in debug mode. This is what we want. It tells you that it'd loading a shared library and is going to disable the breakpoints while it loads. Afterward it enables the breakpoints you've defined. At this point, because our code is dependent on a request, we should make one. I hit a virtual host url that is setup to use our module and immediately see that our method is hit.

Breakpoint 1, get_fullurl (ctx=0x19d4908, use_base=0, args=1) at seoversite.c:330
330             if (use_base) {

This isn't entirely useful information. I know where my method starts. However, we aren't limited to just this information. Let's say that I want to see the back trace of method calls that got me to this method.

(gdb) bt
#0  get_fullurl (ctx=0x19d4908, use_base=0, args=1) at seoversite.c:330
#1  0x00007fce2402299a in filtering (f=0x19cba78, bb=0x19d0298) at seoversite.c:1739
#2  0x000000000044f446 in ap_pass_brigade ()
#3  0x000000000047b2ed in ap_proxy_http_process_response ()
#4  0x000000000047bad0 in proxy_http_handler ()
#5  0x000000000046af37 in proxy_run_scheme_handler ()
#6  0x0000000000467524 in proxy_handler ()
#7  0x0000000000441450 in ap_run_handler ()
#8  0x0000000000441ce9 in ap_invoke_handler ()
#9  0x000000000048823c in ap_process_request ()
#10 0x00000000004851f0 in ap_process_http_connection ()
#11 0x000000000044ae72 in ap_run_process_connection ()
#12 0x000000000044b2f9 in ap_process_connection ()
#13 0x00000000004ac48b in child_main ()
#14 0x00000000004ac56e in make_child ()
#15 0x00000000004acb03 in ap_mpm_run ()
#16 0x000000000042857c in main ()

This is now getting useful. Not only is this nice for debugging purposes, it's really nice for inspecting call traces of modules you've never used before. Inspecting the actual behavior will provide you a much better clarity of the source code. In our case, this confirmed my assumption of the calls that lead up to getting to get_fullurl. Now I know how we got to my method, but I need to see where it's failing. Ideally I have the source up in Emacs and can look at line numbers while debugging. What I don't get with this however is the ability to go one line at a time and inspect the values and types of variables. GDB provides this capability.

(gdb) n
334                 if (ctx->f->r->parsed_uri.query != NULL && args) {

As with most debugging software, this allows me to step through code, line by line, to see what the execution does. Obviously, this is a useful thing. But, to enhance it we get the ability to inspect variables.

(gdb) n
334                 if (ctx->f->r->parsed_uri.query != NULL && args) {
(gdb) p args
$2 = 1
(gdb) p ctx->f->r->parsed_uri.query
$3 = 0x0

And, to make sure you have the appropriate context for what you are dealing with, you can print out a handful of lines to accompany your inspection.

(gdb) l
327             char* baseuri;
328             char* fullurl;
329
330             if (use_base) {
331                 baseuri = get_baseuri(ctx->f->r);
332             } else {
333                 if (ctx->f->r->parsed_uri.query != NULL && args) {

Ultimately this has made things easier. And, there is no longer the "ok, add a new print line in here, recompile, run, check logs, rinse and repeat" type of process. It really simplifies my life when I have to hack on C code and I know it helped one of our other guys track down some memory based issues. The only downside here is that there is no longer any sword fighting.

Permalink | Leave a comment »

Adapting To Change

Sat, 10 Oct 2009 13:32:56 -0700

Homework If you watch the following video with Trent Reznor, it's about 40 minutes long, you'll likely understand the rest of this post. But, for the TL;DL crowd, the interview had a lot of undertones to it that I really thought needed to be elaborated on. Not only do they pertain to what he's doing within the music industry; what he discusses has to do with, at the very least, the tech industry. I see it as a fundamental shift in the way business is being conducted. At any rate, take a glance, and there will be more after the video. Thoughts First- Trent Reznor is the f*ing man. Second- Trent Reznor is the f*ing man. Third- there is new information coming out lately about how we're in a time of mergers and acquisitions (M&A). M&A is a way to grow your company in size (employment), in revenue and, if you did your homework correctly, in profit. Ultimately the point of the larger companies is to make themselves money. They need to sustain the business styles they have grown into. Younger companies tend to be far less concerned about the profit engine as they are a bout their idea. Pay attention here. If you listened to what Trent said about record labels and artists then this should sound familiar. However, his thoughts don't just apply to the recording industry.

Small businesses tend to be focused on producing some idea that changes the world, makes it a better place to live in, or makes my cheese burger taste even that much better. They are _not_ focused on whether the special sauce will be cheaper if they use Angus vs norm. In fact, a lot of smaller companies will make a deliberate decision to go for quality. They know it will cost more, but often that deliberate decision is an important differentiator in the market. However, the businesses doing M&A tend to be focused on things that generate profit. They need to feed the meat grinder and produce dollars. After all, they have investors to satisfy. But, and this is where Trend hits the nail on the head, if you want to keep your vision, in most cases, you need to stay away from the M&A scene. If you're looking to just make some cash or want a much larger exposure to the market knowing that you may lose some of that "somethin', somethin'" then M&A may be right for you. Something to keep in mind here is that the internet does allow for a great "grassroots" style exposure method. While the phrase it overused and overassumed, "your stuff could go viral!!! OMG!" Fourth- in the interview Trent provides some notes about what artists starting out can and should do. This is not unlike the technology industry. Now-a-days there is often very little capital required to start. You can literally buy a domain and hosting for 21 dollars a month, throw up a blog, software, etc. and you're on your way. The most expensive equity at that point becomes sweat equity. Here is something to note, investors tend to like it if you require less money from them and can produce something which will make them rich. Seems to make sense yea? If you can show that you work your ass off, made this product in your "free" time, and then have plans for the future, you're well on your way to getting some funding. Keep in mind that getting funded is much like acquisition. While the investor will be much more likely to care about your vision they are still concerned about the profit engine. However, because it doesn't feed into some conglomerate view of the bottom line, there is a bit more attention on the details that make your company tick.

Fifth- things are changing. The way that business is done is changing. This is somewhat hidden in my fourth note. Trent noted a distinct difference with the way artists need to operate now from when he was coming into the industry with Nine Inch Nails. The technology that exists today makes it so that the companies today that are really taking off are just starting as hobbies or side projects. There are a ton more as well. The point here is that these companies are utilizing the infrastructure to make something really interesting and really useful, but most importantly, something that they themselves want to use. This is where I see technology companies going. There are a few things about this topic I haven't elaborated on. If you make something that you yourself want to use then you will likely lack passion, devotion, and when the chips are down it'll be easier to cashout than to dive back on in. Starting projects, companies, or ideas is hard work. It requires a ton of time and effort. You'll get to a point where it will feel better to just throw it away. But, if what you are doing is worth it to _you_ then you will pour that extra cut of "oomf" and push through it. And, IMO, if investors overlook this crucial element, they don't deserve the profit you would make them. :)

Permalink | Leave a comment »

Video Editing Software for Linux: Kdenlive

Sat, 10 Oct 2009 12:45:09 -0700

Linux has some great software built for it and the video editing arena is no different. The videos that I create tend to be fraps based out of games, but they still require some editing. I was interested in finding some software that wasn't hard to use and would allow me to produce quality videos. Enter Kdenlive. Kdenlive is a non-linear video editing tool that provides nearly everything that an amateur like me would want. And, having seen Adobe Premiere Pro, I'd imagine it's what professionals want as well.

The software itself is laid out really well. My focus and attention is always on the video itself. And, since the second area where I do most of my detail work is with the tracks, it only makes sense that it be very large as well. The project tree, left of the video, can be toggled for other things as well. You can view the effects and the transitions lists in this pane. Then, should you want to add something you can drag it in place. I have checked out some other software before too but it was incredibly buggy and way too crashtastic. The one I tried the most to use was OpenShotVideo and it just isn't there yet. It seems like it could be a great tool in the future. For now, this novice is sticking with Kdenlive. :) At any rate, I'm not great at "reviewing" video software since I do so little of it. I just wanted to throw out a post mentioning the capabilities here in case others of you who do the same thing I do with video were interested in it. I hope that this is helpful to someone.

Permalink | Leave a comment »

Fixing Mistakes - Some Thoughts On TDD/Rot/Debt

Mon, 07 Sep 2009 12:26:17 -0700

Software development is always a good way to realize that you're human. I'm sure there are other professions that give you this feeling as well, but my experience is limited to the software development field so I'll talk about it from that light. There are really two ways to go on this subject. I could talk about code rot and the need to clean it up and/or talk about test driven development. Because I think they both have something to do with each other, why not make it about both eh? The Church of TDD Test Driven Development is a nice concept. The key here is concept. The idea here is that you write tests for a piece of functionality (ideally you know what you're trying to get out of your function) and then write the actual functionality such that all your tests pass when your functionality is complete. Why you might ask? Well, I'll answer! The reasoning is that when you're 3 months into your project, breaking Brooks' law, and trying to cope with the ever changing requirements of your project, you'll simply have to make your change and re-run your tests. If they fail, you caught the problems at test time rather than at 3 AM when your boss friend calls you up and lets you know that he was just informed about emails being sent around to customers with usernames, email addresses, phone numbers, and contact lists for about 500,000 users. Obviously a made-up case, but I have read about similar scenarios. That said, test driven development is only useful when you're testing the right things. Testing functionality isn't just about testing that you get a string back in all cases. Let's say your working with Java and StringTokenizer's. You might need to test the return values of each token. Also you may have to test the number of tokens or if a token is supposed to be empty that it isn't null. If an exception should be thrown, you should check that case too. There are plenty of example cases to check here, but the point is that if you're testing a method and only checking that something was returned then you aren't finished.

Start out with a plan. This works in either the agile or waterfall methodologies. By this I mean, think about what you are about to produce. It usually helps me to take a step back and process what the method is expected to do. "What do I expect to see when I call x()?" "What assumptions do I seem to be making about x() where I'm calling it?" Running through questions like these are a good way to get a base set of tests for your functions. So now you have a handful of tests that should make sure your code works right? Right!?!? Not necessarily. Let's go back to my first line here. "Software development is always a good way to realize that you're human." You'll miss something. It's inevitable and, what's better, it isn't because you are bad at what you do. There are so many other things that are likely going on in your brain. If you are anything like me the whistle of a dart flying by your head or the ping and the pong of a game going on in the distance is enough to bring you out from your "I should write tests" concentration. What's more, there are just going to be cases that you miss. It is O.K. to miss things, but there is a stipulation here. When you do and when it is caught, you need to write a test that checks for the situation/case. Subscribing to TDD for the sake of it wont get you far and will likely slow your productivity. But, if you take a pragmatic approach to TDD and write tests as you think of cases your source base will likely be more stable since you'll find bugs when you run the tests instead of through the client. And, you'll have a system in place for those bugs that do slip through so that you can make sure they don't come up again. Clients are a lot less understanding when you have issues with your code base and you can't assure them that you wont ever have that problem again because you have no tests in place to ensure it. Code Rot and Technical Debt

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite.... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise. - Ward Cunningham on Technical Debt

Software development is one of those knowledge based fields. You write software with what you know at the time. If you are any good at what you do, you should still be learning every single day. The old saying "you can't teach an old dog new tricks" really doesn't hold water in this industry. Once you stop learning new things, techniques, and methods, you've effectively signed yourself into retirement or unemployment. That is just the way our industry works. So, with this new-found knowledge and all powerful ability you'll begin to smell something. It's a familiar smell. You'll be working on your code base and look at a method you wrote a couple weeks ago and have to plug your nose. That, my friends, is code rot. The two, technical debt and code rot, are interesting. Understanding that you may take shortcuts to get a project moving only works when you refactor. If you do not refactor your code after you've introduced the questionable code you're in for a "fun" time. Lets not get the two confused. Code rot is typically when things change external to the code such as your ability to program or changes in the industry that your software works with. Technical debt tends to be what you introduce knowingly to the code with the intention of cleaning up later. However, as you find code rot, it becomes technical debt when you do nothing about it. Technical debt is something that needs to be paid back. Relating this to real life; imagine yourself going so far into debt that you cannot imagine yourself ever getting out from under the load you carry. You will either be stuck with extreme payments the rest of your life or you'll claim bankruptcy. In a software project, the equivalent of bankruptcy is a complete rewrite. You have invested time, sweat, tears, and likely a decent amount of money into whatever you're working on. You don't want to lose what you already have, but sometimes it's inevitable. The point I'm trying to make here is that you need to be willing to pay back your technical debt such that when you incur more debt, you aren't overwhelmed by it. Paying that back can be related back to time and effort. Hell, even if you spend 5-10% of your time just refactoring your code you'll have been paying back some of the incurred debt. I'm probably beating a dead horse here so I'll move on. Problems There are always issues right? "Sure, what you're saying makes sense, but we don't have the time or the budget to go through our code and fix it." I know. I've heard and said this before. But, think about it like this.

I have a car. I drive it places and it serves me well. I put gas in and all is right in the world. Then one day I feel some light vibrations when I press the brake. I don't think much of it because the car still _works_. I don't have the time to get it taken in to the shop. About a week or two later I feel a stronger vibration and every so often some squeaking coming from the car. Again, no time- I'm a busy man! Another couple weeks pass and now the squeaking is a screeching and it's loud. It pierces my ears. I know that sound and realize that it means I likely need to fix more of the car. Now things are getting pricey and I don't have the money to fix it all at this point. You see what's happening? I put it off so long that it became prohibitively expensive to surgically remove the rot. Instead now it is going to cost me a fair amount of cash, time, and effort to actually get these fixed. And, who knows, if I continue on the current path I might end up in a wreck. Still when you are under constraints that you have very little control over there doesn't seem to be much that you can do. Most of the time you are forced into situations where you either piss off the PM's and management or just continue to let the code grow it's warts. Ideally, you'll prove to those that make time and budget based decisions that your changes will be worth it. By worth it I don't mean "the code will be better", I mean "in the long run, we'll say 30% and gain 5% in revenue". Solutions One of the serious mistakes that developers make with manager types is that they think they are relating to them with terminology and analogies. Typically it requires actual impact to prove that you are right. In order to get impact values you have to gather some metrics. Hopefully some of what follows can help there. If you already have tests in place for the code you've written you can do a couple of things. Lie to the people that are asking you what you're working on. *cough* working on that scalable synergistic thing-a-ma-bob that will grow our profits exponentially *cough*. Or, you can tell them the truth. Regardless of what you do, you can have some faith in the fact that your system should operate in the way you expect it to because you have tests in place for just this reason. At some point this code was going to be changed and you wanted to make sure that you did some CYA when it did change. If you don't have tests you can again, lie to the people. Or, you can explain to them why it is important and some tactical approaches to solving the problems. Some fuel for the fire:

Measure and Frequently Repay Technical Debt - Intel Software Network Blogs
Start Paying Your Technical Debt - The Mikado Method - Thoughts of a Goldfish
The IT Manager's Dilemma with Software Debt - Chris Sterling's Blog

Think about things like this. You want to get rid of this technical debt. It's very likely that the project manager does too. But, explaining these things to the people who are interested in release dates can be challenging. Try explaining that customers are more happy when their software works. The more you can ensure that the better. Often it'll require charts and figures with bug numbers over time. Be sure to show where you're making improvements too. For instance, "as you can see we refactored the code on 3/12 and the number of bugs being reported decreased by n%". By doing this, you are effectively showing ROI and how your changes affect the bottom line. For people concerned mostly about money that is important. And, if all else fails, show them this video! Sometimes it's better to just ask for forgiveness in doing these things. Even if it means temporary irritation, the benefits outweigh the downside.

Permalink | Leave a comment »

Beaker - Yet Another Web Server

Sat, 05 Sep 2009 05:28:14 -0700

In the pursuit of learning a new language I picked up Programming in Scala. I had fumbled around the Scala examples and read some of their documents like Scala by Example. I did replicate one of their examples by implementing a Quick Sort. I don't feel it's sufficient to know the language by taking an example they have provided and merely reimplement it, so I've decided to try my hand at a web server from scratch. So far it's been quite fun. At first I figured, "All I have to do is spawn some threads and write to the output stream". Then, my curiosity got the best of me, per the usual, and I thought, what about if I just take the bytes from the files and write those instead of just reading lines from HTML. Then I can support any content types. Obviously just using file extensions wasn't going to be enough for me so I figured I'd integrate mime-util as I've had some good experience with that in Axiom Stack. And, I've been working on a implementing Handlers in the way that allow different handlers to manage different paths. So, now what do I have? I have a functional web server that provides 200, 404, and 500 level response codes. Ultimately I have an experiment that has gone pretty well. To get a feel for whether I was actually headed in the right direction with web servers and Scala I decided to take a peak at the Lift source. While I'm obviously not there yet, it seemed as though I had the right idea in terms of how they thought it should work which is comforting to me (something to keep in mind is that I do have some experience with web servers since I'm an active committer to Axiom Stack). So. While it's great to have a static content web server, that's really just the tip of the tip of the tip of the...stack overflow...iceberg. I'd like to bring this server into a better set of compliance, make it faster, and truly test the "scalability" of the server. I know that it handles me slamming on F5 pretty well but that is the only kind of testing I've really done with it so far. Additionally, I'd like to figure out what it would take to get the distributed requests and such working well within it. I might try to bring in Terracotta for that portion of it. There are a lot of things I'd like to do with Beaker but I just don't know whether it is worth my time to continue to develop the server. We'll all find out together I guess. ;D Also, while all this is great, the biggest reason for doing this is to really get a good personal comparison of Scala and Erlang. While the threading model for Scala is stolen from Erlang, the language similarities essentially stop there. I may try my hand at replicating this server in Erlang at some point.

If you're interested and haven't yet fallen asleep you can grab the source for Beaker on Github. Fork it, hack it, let me know what a noob you think I am. I can handle it. Enjoy. *Update 1* I suck at art. No making fun of the beaker - it's hawt (on second though, maybe I should lace it with bacon...).

Permalink | Leave a comment »

Foreplay and Programming Languages

Sat, 01 Aug 2009 17:08:49 -0700

Programmers will tell you that you should learn a new programming language every year. I'm not going to tell you otherwise. In fact I agree with this. Here's why, I believe that a language is just a tool to get your job done. More over, Python is not how you program, Python is a means by which you write programs. When I got out of college I didn't really grok this. I knew there were many different programming languages, but Java was how I programmed. When I thought of a problem, I saw the solution in Java. It's almost like communicative languages. If your first language is English and you are taking French, you translate the French to English before really understanding the French. This is what I would do with other languages and Java. It's not the best method to learning the language I've found. In fact, every language is different so trying to transpose one language to the other tends to push the write Fortran in any language approach (no scientific study, just my experience). So, if I take a Java programmer and give them a project and tell them that the language being used is Javascript, they will write Java and the code will likely get written, but poorly and need to be completely refactored. Going For The Goods

The way I would approach new languages was to take some existing problem and attempt to solve it in the new language. While this is one of the steps you'll need to take to really know the language, shouldn't you get familiar with it first? Think about your education for a moment. When you were in school taking Spanish, did the teacher start you off by saying english("I") and then spanish("I") or did she start you off by saying spanish("Take out your translated version of Tale of Two Cities in Spanish and read")? I'm going to assume the first scenario as this is what my experience was. For me, diving into an existing problem and trying to tackle it with a language I've had no experience with just ends messy. It's almost like when you...ok I'm going to try not to make the Rails mistake. But, really, everything we as humans do when learning is experience based. We crawl, then learn to stand, then walk, then run...and so on. So why not start out with the same approach and build momentum for your language? I'm not saying that you need to spend hours relearning what a for loop is. I am saying that you should spend some time getting familiar with the syntax and differences of one language before trying to "go for the goods". The Foreplay Because I noticed that I was trying to run before walking with languages, I started taking a new approach. Instead of transposing language constructs, I instead started to rewrite common/well known data structures or algorithms in the language I was learning. There are a couple reasons to do this. First, I constantly feel out of practice when it comes to algorithms/data structure design. I don't claim to be a math wiz so re-reading and implementing them is great practice. Additionally and perhaps more importantly is that the algorithms/data structures I'm implementing aren't new ideas. These are tried and tested concepts that span languages. Thus, I will have comparable results. Take this for instance. If you write a merge sort in one language, you'll most certainly be able to write it in another language, it will just be written differently using the appropriate syntax and language designs. Since you already know how a merge sort jiggles when you poke it, you can compare your version to the typical. Plus, you can calculate the speeds of doing said algorithm in the new language. This will provide you with some benchmarking for better decision making about the language. Another Analogy

If you need another analogy, I used to play instruments. Typically, outside of August Rush, people who haven't touched a musical instrument must go through some period of time to learn the instrument. You need to learn how to caress it so that it will make music rather than just a mixture of noise. When you start out you don't learn Für Elise you start with notes and scales. The notes and scales apply across instruments. You have to learn them for the instrument you're using, but you understand the concepts. Conclusion Warming up to the language seems to work best for me as I can compartmentalize the diffs of the languages preserving my memory for other interesting things. Then I can pull out what I need from each language. Think of it like this, the algorithms and data structures are your minds libraries and the syntaxes of the languages are the methods written on top. The algorithms aren't likely to change, but you're trying to evolve your syntaxes. To summarize: I have a specific methodology for learning languages that I've found works really well for me. Although I'm sure many of you will have your own method for learning them, maybe this will help someone trying to get more familiar with differing languages and maybe give you a different approach. Disclaimer Always leave room to learn more about algorithms. Who knows, you might be the one to figure out how to make quick sorts obsolete. So, don't write them off, but know that they are tried and tested and because of that, produce predictable and comparable results. That is my reasoning for recommending them.

Permalink | Leave a comment »

Implementing Java Interfaces from Javascript with Rhino

Fri, 17 Jul 2009 01:42:23 -0700

Rhino is a fairly awesome tool. You get the full capabilities of writing Javascript, plus the availability of the Java ecosystem. However, there is another awesome feature that I didn't know about until a few months ago when I read Todd Ditchendorf's post about thread synchronization in JS. You can actually implement Java interfaces from your Javascript. For example, let's say you want to override the toString method on a String object. I used his technique when I was writing a module to allow HTTP calls from the JS side of the house, on the server side. Below is the code that allows for asynchronous requests to be made. Very simple and very much like Todd's examples, but that is all it needs to do and it works beautifully.

...
ajax: function(url, data, callback, type) {
  var r = new Runnable() {
   run: function() {
    type = type || 'get';
    callback(axiom.HTTP[type](url, data));
   }
  };

  new Thread(r).start();
}
...

In essence it creates a new Runnable object, defines the run method, and then passes that Runnable object to a new Thread. While this code demonstrates the functionality with the Runnable object, you have the capability to do this with any other interface right from within your Javascript. It's really handy and took me a while to find so I thought I'd post something about it. Hope it helps.

Permalink | Leave a comment »

Git: Tying a Remote Branch to an Existing Local Branch

Tue, 30 Jun 2009 16:03:49 -0700

As I was cleaning up my code directory and forking projects to have personal dev branches up on Github, I ran into a problem. I needed to tie a remote branch that I had created to an already existing branch. So, I had branch already in my list of local branches. I did one of these: git push ncb000gt master:refs/head/branch This made the remote branch and then a: git fetch ncb000gt Pulled it into my view. Now I have two branches, one local, and one remote. I had changes and commits in the local one that I didn't want to lose. I could have tried to solve the issue following the .config method, but instead I decided to do it a different way. Basically by moving/renaming the existing local branch: git branch -M branch branch_temp Then, create you local branch that is tracking the remote branch and check it out: git branch --track branch ncb000gt/branch git checkout branch Then all you have to do to maintain the existing commits is merge the code in. git merge branch_temp That should be about it.

Permalink | Leave a comment »

JSConf 2009 - Where were you?

Sat, 23 May 2009 18:07:02 -0700

To say that JSConf 2009 was a whirlwind isn't giving it's creators enough credit. A few ways I'd describe it are action packed, intimate, and collaborative. From talking with Chris Williams, that was the goal. The Overview

Obviously, as the people who were at my talk noticed, I'm not an artist. But, this gives you a bit of an idea of how the conference was laid out. To the left you have the main presentation room, Track A. To the bottom right you have the Track B room. And, in the upper right hand corner was the Hack room, aka the Hack Track. The point of Track A was to present some information that would take a much longer amount of time to really get across. Ideally the presentations were on a single topic and lasted about 30 - 50 minutes depending on how much room you wanted to leave for questions. Track B was to be able to present on some topic that didn't need a full 30+ minutes. The idea here was to have more of a discussion format, hence the single large table that allowed for everyone to face each other. Then, the hack track was to encourage people to sit down and just code. The two twinkies/equals sign represent the food area. All I have to say is bacon...nuff said. The food was excellent to say the least and no, there were no twinkies...I'm just a horrible artist. There was one track that I couldn't really show on my diagram. This track was the Significant Other track hosted by Laura Williams. The Significant Other track was a way for us techies to bring our...you guessed it...significant others, but not let them sit in a hotel room or staring at a screen of geek speak. Instead these people were taken around DC to do the touristy thing, checking out the monuments, the White House, and so on. Then they even had a shopping trip planned in Old Town Alexandria. No other conference I've been to had this _feature_ and nobody I've talked to has had that either. /cheer When I say I was impressed with everything that Chris and Laura did for this conference, I'm not giving them enough credit. Interesting From what I was hearing in the crowd, the topics that people brought to the table in both Track A and B satisfied everyone's appetite. The topics ranged from client side games in JS to Objective-J to server side JS. There were topics on ORMs, distributed browser testing, and data sources. I also recall there being a Track B session on creating midi's with JS. Because I am interested in everything, I like learning, I'm just going to pick a few talks that haven't gotten much in the way of publicity yet that I found fairly intriguing. OpenGL and Real Time Simulation Andrew Turner spoke about some work that he and his company have been doing with integrating JS and really high powered graphics engines. As any of you who know me know, I'm really into in 3D engines. Some of what Andrew and his team have built is the ability to create objects within the worlds they've developed by using nothing more than JS. You can control many of the elements in the engine with just JS. This means that those people who were originally deemed "just web developers" now have the ability to join teams of game developers/designers to really have a hand in the outcome of a game. While you could use most any scripting language, the number of people who know javascript is quite vast and the concept lends itself to that mass of people. I couldn't find Andrew's slides anywhere. If you can, send me a link. Client Side Gaming Games on the client side are an interesting subject in and of itself not only because they are games, but because of the challenges presented. John Resig talked about his approach to making client side games with Javascript and some of the issues. One of the issues and solutions that I really found interesting was his approach to tackling cheating. If you take a game like World of Warcraft, much of the game is controlled by the physics, money, and items you carry. Imagine that you had an interface to give that mighty sword of killing a +300 to damage. While that doesn't exist in World of Warcraft, most web people use a tool called Firebug. So, we do! Suddenly we can't rely on the client (browser) to give us accurate information about the user. This dilemma means that we need to have the client pass us specific information however, anything other than "attack" or "move" should be controlled on the backend. John also talked about the issues around a game he's working on called Deep Leap. The approach he has taken here to curbing cheating is by replaying the game on the server side. So, in essence he records all the "moves" and pieces that exist and then uses the same JS on the server side that can be done on the client side. This is a really great idea. You are using the same code base which means you only have one bit of code to maintain and it's a really simple way to remove cheating. :) Take a look at John's slides Server Side JS There were a few presentations about SSJS. Kris Zyp, James Duncan, and I all gave Track A talks on our respective frameworks. And, Kevin Dangoor held a Track B discussion on what the #ServerJS group is doing to standardize SSJS. I don't bring this up to make a plug for the framework I work on. I bring this up as just an example of the number of frameworks popping up with some power behind them and to make mention of the stuff the server JS group is doing. The #ServerJs group has been working to standardize libraries and modules that plug into SSJS frameworks. One of the issues plaguing the SSJS world is the utter lack of standard libraries. Python, PHP, RoR, etc. all have massive communities behind them because they have huge library sets. But, most of the JS world has been client side till now. So where are the libraries for reading files? What about just the ability to pull libraries in a standard way? None of this has existed till now. So, while we, the framework developers, go about making these tools for people, we can't really expect anyone to switch over without some basic level of library support. If you are interested at all in SSJS I highly recommend taking a look at the Google Group and contributing to the discussions. View Kris' presentation on the Persevere framework Take a gander at my slides from JS Conf on Axiom Stack I couldn't find James' slides anywhere. If you can, send me a link. Others Some others that I wanted to mention include Titanium App by Jeff Haynie, Real Time Web with XMPP by Jack Moffitt, and CouchDB to the Edge by Chris Anderson and Jan Lehnardt. Fin JS Conf 2009 was a flurry of information and great ideas. To say it was awesome is insulting. Hopefully, I will see you all at JS Conf 2010. Till then, keep up the good fight.

Permalink | Leave a comment »

WoW: Versioning Your Addons

Mon, 13 Apr 2009 23:13:11 -0700

If any of you are like me, you play World of Warcraft and love your mods. I try new mods all the time in the attempt to better my UI. I tend to look at it as a constant work-in-progress. So, I'm trying to find mods that best fulfill the needs I have in the game while minimizing the overhead created on my CPU and RAM. To go along with my addiction to addons, some of the people I play with have the same mentality I do towards mods and constantly tell me to try new mods. The problem is that you never know if the mod is going to conflict with your setup. Also, if you are upgrading to beta versions of addons you need a way to just rollback. Normally this is done by copying your SavedVariables directory out of your WoW directory. This is tedious. Enter Git. Git Git is a version control system generally used when trying to save time and effort on source code. However, it's also useful on files that have settings. Essentially what version control software does is allow you to make changes to your files, and then recover older, working sets when you completely annihilate your files. Believe me, it happens. Let's focus on just Git for now to give you an idea of how it works and we'll apply it later to WoW specifically. There are a lot of terms that go along with Git. Much of which are irrelevant to the masses. I'll explain some of the basics. A repository is the directory that all your Git information is stored. So, if I was in my WoW directory and typed "git init", that directory is now a git repository. If I was in my Addons directory and typed "git init" I would have made my Addons directory a repository.

Branching is the notion of creating copies of the repository. These branches store the differences between your sets of repositories. When you create your repository, you are actually creating a branch, among other things. The initial branch is called "master". You create branches by typing "git branch [new name]". This will branch the current branch you are on. So if you are currently doing things within the "master" branch, you will create a copy of that branch. To see which branches you have and which one is active, type "git branch". The branch that is starred is your active branch. When you switch between branches it is understood as a checkout. So, lets assume that I created a branch called "experimental" (git branch experimental). I am not immediately in that branch, I still have to move to it so that I don't destroy my stable state. I would type "git checkout experimental". To save your changes you commit. This means that you store your changes into the branch you are working in. You do this with "git commit". Now, know that git doesn't necessarily know about anything new you are adding to the branch without your explicit say-so. Thus, you'll need to type "git add .", where the '.' is the current directory, and then 'git commit -a -m"Saving mah changes!"'. This will put all your changes into your branch. At some point you'll realize that your experimental branch is your preferred setup and you'll want to make it your master branch. At that point you'll want to merge your changes. This is easily done with "git merge experimental master", where 'experimental' is the branch with changes and 'master' is your stable branch. One small detail that I neglected to mention till now is that Git runs in a command line. "Oh noes!" Easy! Deep breaths. It's ok. There are a few ways to tackle this beast (aim for the legs!). You can use the Windows installer for Git or Cygwin. Either will work. If you are using Linux or a Mac, get your terminal on. For the purposes of this I'll go through it with the Windows installer for Git. For the rest of you the commands above should work assuming you installed Git correctly. Open "Git Bash", should be on your Desktop. Once there you should see a command line window (black screen, white text and a blinking cursor). This is good. Once you are here, any of the commands I described above should work. I could go on for days describing the details that are Git, but this should be the necessary information for your success. If you are interested in learning more, check out the Git documentation. Git with WoW Now that we are past some of the basics, lets get into how you can use it in WoW. When I'm working with new mods/addons/etc. I first go into my Git command line (see above). I change into my World of Warcraft directory.

cd c:
cd Program\ Files/World\ Of\ Warcraft/

Once I've moved into my WoW directory, I need to decide what is important in terms of layout. We have our Addons directory inside of the Interface directory, and we have our settings inside the WTF directory. I'm going to change into my settings directory. cd WTF/Account/AccountName/ This puts me into my actual account. From here, I want to create a repository because everything here is important to know about when it changes. Note, Auctioneer type data isn't necessary to know, but will be included here if you use the mod. So I create my repository and then create a branch of the data so I can muck around with settings and not disrupt the stable set.

git init
git add .
git commit -m "Initial commit to stable branch"
git branch experimental

Now that I have my stable and experimental branches, I can move between them by checking each of them out. git checkout experimental or git checkout master The above allows me to move between branches without any issues. So, if you've gotten this far, that means you're now ready to version your "settings" files. But, you now need to make sure that you keep the versions going so that you can roll back anything you might mess up later. Let's say you've been gaming for a while and found that your unit frames sucked being right in the screen or that grid was good enough for party frames. So you make some changes to some mods. You need to make sure you commit those changes back into the branch you are working in. git commit -m "Removed party frames, grid is enough." Now you're cooking with fire? Welcome to being a mage... You're done with your settings. Congrats, lets move to Addons. I'm going to assume that you are in your WoW directory (see above if you've forgotten). cd Interface/Addons You need to repeat the steps above.

git init
git add .
git commit -m "Initial commit to stable branch"
git branch experimental

You can now switch between branches and add/remove/modify mods. When you start getting into this, note that you don't need to commit every time you leave WoW. It's just not necessary to track every settings change unless you feel you need that level of control of your UI and the settings within.

My Approach I usually run on my master branch for a while and then when I get the urge to change the UI spend a few hours doing it. So, what I tend to do is this:

Run on 'master'
Create a branch: 'experimental'
Work on my experimental branch till it's finished. This includes removing addons (git rm XPerl*).
When I remove items from my UI I usually commit the changes I'm making and state why. This allows me to be able to switch between branches should we be raiding or if I need to run an instance with someone.
Once I'm done making my changes and am happy with the outcome, I merge the experimental back into the master.
Rinse and repeat.

There are uses for Git here without just massive UI modifications too, but I tend to really use it for that. How do you use Git with your WoWing? Are there any things I missed that you prefer to do? Let me know. Also, let me know if something doesn't work for you.

Permalink | Leave a comment »

Scratching An Itch

Sun, 12 Apr 2009 12:48:52 -0700

A couple friends an I were talking a few weeks ago about building applications. There are a handful of reasons for creating websites or web apps. The topic of discussion that day was about applications that do something you need and not just about the ideas, but about finishing those projects. To say that creating applications to scratch your own itch is fun is an understatement. Usually you are heavily invested in the idea and so you have a lot of drive to start it out. I love the idea stage of ideas. I have a lot of them and have a whiteboard in my apartment covered with ideas and mini-architectures around those ideas. Yet, most of them don't seem to happen. I know that I am pretty well motivated to create stuff that I've bought into, so what are the reasons that these projects seem to stay stagnant in the idea phase. Exhaustion. The first killer is exhaustion. The reason this ruins the drive to do more is that when I get home from work where I'm trying to solve other problems all day, I'm not necessarily focused on pushing through mine. Doing more work on top of the work I've already done doesn't sound like loads of fun to me.

Ideas are fun. Plain and simple, work is hard. Running around thinking "I can make it fly and produce magic ponies too!" is great. But, when I'm the one that has to produce the result of the idea phase, it becomes a major stumbling point to getting something done. Broad ideas are broad. Without defining a limited scope to what you ought to be doing, your focus and ADD will take you in 90 different directions. Rather than getting something done, you'll be focused on starting a number of different things and leaving others to hang out in a partially working state. Lacking skillset. When you are lacking a certain something to get the project finished, you will find it very frustrating to not be able to just hammer on the rest of your idea. So what do you do?

Focus. Harness that N.A.D.D. and get something created. The original idea doesn't need to be your "pony, dragon, rocketship" release. It's your proof of concept (POC). You need to see that your idea _can_ work. You being the brainchild behind it, make it work and then add onto it. Give yourself a deadline. When I want to really get one of those projects on my whiteboard done, I give it a deadline for a POC. This means that if the idea isn't done by that date that my interest either isn't there, or that the problem is far more complex than I originally thought. If the idea is too complex, I'll know early on and will have to rethink my approach. Otherwise, I should be able to push through. And, generally by applying a date to it I know that my interest level is there. It's just a matter of doing it. Have people to push you through the "hard times", as my friend called them. The hard times are when it's not easy going, when you need to sit and actually think about the solution and the best way to implement it. Often times when the hard problems hit there is no obvious solution. This makes it quite a bit more complicated and if you can have someone around to get you through those points, you'll be more likely to succeed. The point here is that you have someone or a group of someones to hold you accountable for moving the project forward to completion. Believe me- this doesn't always work, but when it does it is a good way to keep yourself on track. Also, it will allow you to get feedback about what you are doing. To steal an idea from the agile methodology, "Release early and often". Meaning, put your project out there sooner rather than later and make smaller incremental changes. There are _many_ benefits to this practice, but those are better left for another post at some point. Lastly, if you don't have a skillset, you are fine. Just keep in mind that you might not be able to get some part of the project working for a bit. Either read up on it and learn it or outsource it. Phone a friend, check in the freelancing sites, etc. I know that personally I am _horrible_ with design. If you want squares with black and white, I'm your guy. The point here is knowing your limitations, and finding a way to push through them. This tends to be my biggest competitor to actually producing a full project for public consumption. Now, if I were in your shoes I'd be thinking, "What do you know? You've got a list of untouched projects." Although that is true, this is what I'm using to start doing to tackle that list of projects. I'm curious to know how some of you hunker down and get your stuff done. I know that some people use Backpack and/or Basecamp. I know others use events like Hackday to get their ideas done. I've tried those and they didn't work well enough for me. The Hackday approach is what has worked best for me but there are tools that are other tools helping me out here.

The first tool helping me is Git. This version management tool allows for some very nice things. If you haven't taken a look at Git yet, do so. You'll be glad you did. Another tool, based on Git, is Github. While Git itself is great, having a central repository is very nice for having your source be always available. I've been putting all of my projects up there and while the decentralized model of DVCS is still alive and well I know that I will always have access to the source so long as the internet exists. Lastly, telling people about the projects is the key to me getting the work out there. I have people to push me through those hard times and friends that can help me to complete the design work that I seem to be exceptional with. Back to that conversation I was having with my friend; one thing I realized in that conversation is that usually when you have smaller projects or ideas they do not require a team of people architecting a larger solution and in fact that collaboration can hinder your progress. They require a few hours of attention to just focus and hammer down on. When I started doing that, a few of my projects started to take shape. You just have to dive in and see where you come out. You don't need the communication overhead produced from more people on the project. Ultimately, I don't know that this approach is going to work in my attempt to get some projects done, but so far so good. I've thrown a bunch on Github and am going to be displaying them here as well. I'll be looking forward to some feedback. The licensing for now on most of my stuff is AGPL, but that could change shortly. The reasons for this are all the topic of another blog post about licensing.

Permalink | Leave a comment »

New Host, New Posts, and the Inbetween.

Sun, 12 Apr 2009 03:34:11 -0700

Any of you who recently came to the site may have seen a "Godaddy" landing page. I've recently moved my hosting to Slicehost and am managing the domain with Godaddy. It's cheap and has an easy to use control panel. I was tired of managing my domains with a number of different services. Thus the move. Sorry to anyone that I inconvenienced with that. :) That being said, I'm going to be writing a bit more and about more topics. Normally I try to keep it business oriented but I'd like to get more into the CS side of things and talk a bit more about random things that are coming up. I think that doing that will make me more apt to write. Also, I'm likely going to give this blog a face-lift as it's had the same design since I started. So, look for more soon!

Permalink | Leave a comment »

Be Vewy, Vewy Quiet! I'm Hunting Bugs.

Tue, 10 Mar 2009 02:48:47 -0700

There comes a point in every project, when stuff blows up. This could be your code, someone elses code, or could be a mix of the two. The traditional method to finding problems is a mix of guesswork and print lines. Ultimately that is crap. Better debugging methods exist. Bugs are going to find their way into every codebase. It's just the nature of the beast. Most people are not working on simple HTML and CSS anymore. People are building complex RIAs for the Client or sites that have some incredibly convoluted business logic. Making sure all of these moving parts work well together is something you should be doing on a regular basis. There are a few stages to this whole idea behind debugging and testing. TDD isn't just a methodology, it has some merit. In theory, you write your tests first so that you add functionality that makes the test pass. That being the case, you should be able to run your tests at _any_ time and have them pass and if they do not, you fix the code. This works really well with short iterations of dev cycles. We've included a way to do testing within Axiom Stack. This is done using a tool called JSUnit, there are two, we're using the one linked as it is not dependent on the client side. What we've done is allowed you to write test suites within your code that are automatically picked up. When you create a prototype you can create an object inside a JS file called _test. Your setup/teardown methods, and actual tests go in here. For instance:

_test = {
setup: function() {
app.log("Global Outer Setup");
},
teardown: function() {
app.log("Global Outer Teardown");
},
app_suite: {
setup: function() {
app.log("Inner Setup");
},
teardown: function() {
app.log("Inner Teardown");
},
test_app_getName: function() {
Assert.assertEquals("test_app_getName failed", "test", app.getName());
}
/*and so on/*
}
}

This code is then automatically processed by the test suite system. Now, what good is a testing system if it can't actually tell you what is not working? We've enabled our testing system to write out a results file. This is in the normal JUnit format so that the file can be used within continuous integration systems such as Bamboo. If you haven't yet taken the plunge into continuous integration systems then I highly recommend it. Using a continuous integration system has saved me multiple times when I commit extra code that was unintended and not stable. The system caught it, emailed me (as I farked it up), and I was able to undo my gunkery. CI is great, use it. The next stage in this saga is debugging. Why limit yourself to:

System.out.println('this %^*#@$ bleeeeeeeeepity BLEEEEEP');
System.out.println('this %^*#@$ bleeeeeeeeepity BLEEEEEP1');
System.out.println('this %^*#@$ bleeeeeeeeepity BLEEEEEP2');

We've all been there and had to do it. But, isn't there a better way? Yes Timmy, there is a Santa Clause! Most modern IDE's allow you to run through your code. Axiom Stack is no different. We know that you work hard to make things work well. Thus, within Axiom Stack we included a tool to help developers called the Inspector (I know really original, right?). This, like most IDE's allows you to run your code and step through to see where your mishaps might be.

When you debug you should be methodical. Remember that whole "scientific method" stuff you learned back in highschool? Let's bring it back. We're talking about that whole hypotheses and try-try-again sort of thing. However, the hypotheses is the important part here. When debugging you should be making educated guesses about where your problem lies. To make these guesses you should use the resources you have available. Make sure you review standard error messages and logging. First, you need to have a good logging system. Second, you need visibility of those logs. Log4J is a great logging library for Java. You can use tail, in any Linux/Unix environment including Cygwin, to view the logged information. And, once you have these you are armed for some proper debugging. Now, another thing that helps is if the system you are using has a good way of reporting errors. One thing we've realized is that our default error handling isn't nearly as useful to developers as it could be. So, we're revamping the error display system to provide you with better information. These things combined not only make it easier to learn the system you are working with, they make it much easier to rapidly develop applications and sites. Most of what I've talked about works outside of Axiom Stack, we've just made accessing it a bit easier.

Permalink | Leave a comment »

The Code Makes My Eyes Bleed

Wed, 12 Nov 2008 02:18:23 -0800

Not often do you stumble into projects that are pristine. I certainly haven't anyway. Almost every project I've been involved with has required some degree of unfarking to bring it to a close. But, surely there are things that we can do to make it so that those who have to come in and "finish" don't have the undying urge to gouge their eyes out with rusty spoons?

In general, documentation is a good idea. We all know what I'm talking about, but why don't we do it? It takes time, it's not maintained, and who really reads it anyway? Everyone reads it! Assume that anyone will read the comments you make in your code. It's a very easy way to get inside someone's head. Today I was trudging through some code and had to harass our Sr. Engineer countless times to figure out why was there. Half the time I asked he said, "hmmm...I don't know." It then took the two of us cruising the codebase to figure out the purpose.

The primary argument against documentation seems to be that the code should be readable on it's own. News flash- it never is. I may be the brightest coder in the world, I'm not...but go with me, but trying to read the code that other people produce can be painstaking. Hell, I have a hard enough time reading the written language of some people, let alone code. So please, document what you're doing. You'll save everyone time. There are many ways to accomplish the same task. Know this. My solution for a specific problem and yours may be completely different, in fact I would bet that they are statistically more likely to be different. We all have different backgrounds and standards so things will be different. But, that doesn't mean that you have to ruin any hope that someone can jump into your code.

I've worked on projects where the code was like a never ending maze. Sure there are languages that allow you to do some pretty cool things. You have functional languages, recursion, delegation, list comprehension, etc. and all of these are incredibly powerful. However, making your application only work by passing function pointers around confuses the hell out of some great developers let alone the ones that are likely to maintain the code! Be kind to others. If you're not you'll likely be sitting on tacks tomorrow. How would you like to wake up tomorrow and be called "a_32"? Ok, then you're weird. I'd be a bit unhappy. I'm also unhappy when I see code riddled with that. How long does it take to type arrayList32? Honestly. I hate the numbering and all that too, but that's just irritation. Make your names a bit more descriptive. Nobody is going to be able to take a codeset riddled with alphabet soup and make sense of it. if it's a Hashtable of Sessions, use something like "session_table", or something similar. We all have to get better with these things. I know I do. But, think about the people that are going to be fixing the problems you create today and remember that they likely know where you live at least 8 hours of the day.

Permalink | Leave a comment »