Greg Wilson has been posting interesting things about SIGCSE'08, the conference of the Computer Science Education special interest group of the ACM.

The most interesting of the many many posts is one on the struggles of new graduates, relating a presentation given by Andrew Begel who researched what new graduates got up to when they start their jobs as software developers.

I haven't had especiallymuch experience in terms of identifying and working with new graduates - generally the people I've worked well with and hired already had quite a bit of experience before we met.  But I love the idea of helping someone navigate that early part of their career.

There are two classes of mistakes that happen. 

• The first is that the new graduates need to identify those people that they can confidently interact with and learn the unwritten rules of the environment from, as well as bounce their own ideas off of to get feedback. 
• The second is where the  environment puts new graduates into situations where they can't rely on their existing skills.

As noted, new grads generally get put onto existing projects which have their own politics and problems - usually that the project is late, is over budget, has been poorly managed, has been poorly documented, and, often, has been poorly developed by people that no longer are on the project (and, often, no longer at the company).  Not exactly the smooth introduction to the rewarding career as a software developer...

While they may know the basics of version control, issue management, and so forth, they often have to blindly fight through many unwritten rules that apply to this company and this project.  Each person they interact with is going to require different things from them and also hold information that they might require - they're unlikely to have been taught the soft skills necessary to firstly determine clearly what everyone wants from them, and to communicate what they will do and what they need from others.

I'm tending towards thinking that new grads and those new to the sort of environment you're in (say, worked in industry but not academia, or worked in small company in-house development but not large-scale financial services bespoke work) need a "big brother" assigned to them formally to teach them the ropes and through which any feedback on their performance is handled.

This way the new grad has a single person telling them the good and the bad, in a constructive and trusted manner.  The new grad knows who to ask for help when they need it - whether it's where something lives or how best to interact with the various personalities in the team.

This still leaves the problem of how to identify and train this "big brother" - but that's a problem to be pondered another time.

The Cape Town SPIN (who don't believe in permalinks for meeting information, so you can only ever get information on the next Cape Town SPIN meeting) is meeting on Wednesday, March 19, to discuss Project Automation and Software Configuration Management.

Software Configuration Management

Software configuration management, and its central theme of version control, is and old topic, and yet many people ship software products without really getting all the potential benefits. In this talk I'll survey the current state of the art in tools and thinking. A few recommendations will be made, especially for small teams who are not too keen on heavy-weight process.

Project Automation

Performing repetitive tasks is tedious, error prone and never fun. Software projects is full of potentially boring but important task, such as building, testing and deployment.

Automating such tasks does not only relieve pain but also provide the development team with a useful early warning system. However, one needs to be pragmatic about the time spent creating and maintaining automated tasks.

These are topics close to my heart (and memory, since this is what I've spent a large chunk of my time focusing on the last few months), so I'll probably be at the Bandwidth Barn from 18:15 next Wednesday to hear what the speakers have to say.

As I mentioned a few weeks back, the SynthaSite team were taking a few days break from the usual schedule to come up with a shared consistent view of what it is we are building and how we want to build it.

It was probably the last time we'll ever get so much dedicated time to work together as a team, as the process of some of us moving to San Francisco has begun.  I'm not sure everyone appreciated that (I don't think I did), but I was impressed at the results of our process of hiring the best and most passionate - the end result was something we can be proud of.

We first went back to school to learn the theory of our Agile process (initially vaguely Scrum, but becoming more Scrum-like, especially after this discussion), and applied those theories to made-up projects after breaking up into smaller teams.

After dreaming up products (ours, the "Pocket Rocket", was a jetpack, although it seems it is either a vibrator or moped in real life), we came up with a product vision for our projects using a common elevator pitch formulation - the target market, the problem they have, what the product is, who else is trying to solve the problem and why you're better than they are.

The Pocket Rocket had two markets - the jetsetters (that one never gets old) and the military.  It's for getting from one place to another on your own schedule - unconstrained by traffic, surfaces, and so forth.  (The other team had a much more realistic product.)

We then practised creating user stories for our products - in the form:

As a stakeholder-type I would like some feature so that some reason

After that, we explored two methods of prioritisation - the Kano model and relative weighting.  The Kano model primarily allows you to determine what class of value a particular story falls in - based on the effect caused on your user base if you don't do it, if you do it, or if you partially do it.  Relative weighting allows you to assign relative weighting in terms of both value of the feature, and the cost of the feature.  Using these, you can get numbers out on what the best value for cost features are.  Neither is useful in isolation, but in combination can allow you to develop the features your user base demands rather than simply wants, and capitalise on those things that would be quick wins.

Finally, we played some planning poker - we each got a set of cards (0, 1, 2, 3, 5, 8, 13, 20, 40, 100 - the above set is a Crisp Planning Poker deck), and as each story came around for estimation, we showed the card that represented our guess at the effort involved in doing the work, relative to other tasks.

After that, and a break, we set about doing it again for real on SynthaSite (everything - the site builder, the web site, tutorials, support, our development and staging environments, &c.).  It was a lot harder initially to come up with proper user stories and to estimate, since it was a lot less abstract than our example projects, and we were a lot more passionate about the real project.  But, after a while, we got into a rhythm, and we came up with something that both looked and is really impressive at the end of the day:

Working with a passionate team and being on a journey of incremental and occasional radical improvement on a relevant product is a Big Thing in the list of reasons why I love working at SynthaSite.

While I'm not (yet?) a developer on anything visible to the end-user, I am the first people outside of the development group to see what's been developed.

My responsibilities kick off directly after they've committed the code to the repository - I babysit the process of making clean builds and shepherding them onto local and remote staging servers, and finally onto live.  I obviously need to kick the tires on the build after deploying to make sure everything is fine before passing it on to QA and while doing that I get to see the new features.

Of course, there's little by way of surprise - everyone in the company hears about what the rest of the company is doing for the three weeks that make up our iterations.  But there's a difference between hearing about how cool it would be if we had widget toolbars and seeing how it makes the whole process much easier.

There's been some very exciting stuff happening in informal and formal meetings once or twice a week for the past month or so that I think will make the whole process of using SynthaSite more intuitive and effective.  We've just had a formal meeting where some mockups were debuted to most of the company, and it will head off to the design people, then the front-end developers, and then I'll be the first person outside that team to see it again, as we deploy it to staging for further comment.

One problem we're trying to solve is making this process more open - being able to tell people outside of the team - the whole company, our close friends, our business partners, and the general public - what we're working on, and get feedback.  There are obviously a number of factors to be considered when you do this.

One huge advantage of an agile development process is that one can be pretty ruthless about dropping things that can't be done well in time, or that turned out to be a lot more problematic than it is worth.  Saying we're doing X and then not ending up doing X is likely to be worse than never saying you're going to do X.  And it's really not worth losing the gains of the agile development process - nobody wants to be forced to do work rushed and after hours because someone announced something that can't reasonably be achieved.

On the other hand, talking about X has the potential for comments from the userbase and from opinionated other people, which may mean the difference between a somewhat nice feature and a great one.  It may spark interest from third parties via blog posts or getting on a social aggregator.

It's not a simple problem to solve, but hopefully we can find a way to mitigate most of the risk of being open.  Suggestions are, of course, very welcome.

While there are probably hundreds of people employed as Linux systems administrators, and hundreds more people using Linux at home for fun, in Cape Town alone, our Linux User Group, CLUG, isn't growing along with these numbers.  It should, though.  It provides two talks a month, on weekday nights, by some of the best people to talk about them, on both beginner and advanced topics, on programming to administration.  And it also has a dinner afterwards for people to meet and learn more about each other and help each other.  CLUG meets on the second and last Tuesday every month, with roughly 20 people attending the talks and about 10 people going to the dinner afterwards.

Growing the pool is making new connections, bringing new people into the community, providing new people to learn from and new opportunities for work or play.  It isn't necessarily about bringing new people into the field (but it is a by-product) so much as it is about making everyone in the field more aware of each other.

Since the beginning of the year, I've been trying to distill some lessons from some of my failures on KnowledgeTree.  Part of this process has been doing a lot of reading about development - to make sure that I'm not seriously out of step with my underlying beliefs, and to see how I can improve on what I've done before.  Most of this reading, strangely enough, about project management.

I've reread the staples on the subject - The Mythical Man Month and Peopleware - with a deeper understanding and finding greater resonance in this reading - I assume due to increased experience.  And I've also read some more recent texts; the latest is Scott Berkun's The Art of Project Management, which Brad recommended.

This isn't a review of the book, but I feel I must say that this book is probably the most succinct instruction book on how to get a team to deliver a project, no matter what technologies they use or what development methodologies or business processes exist.

I like how Berkun starts by defining his use of "project manager" as being "whoever is involved in project leadership and management activity", and offers that some people might want to translate it as "person thinking about the project at large".

To me, it seems the project leader is the person who lives the project - the one who has the clearest idea of what the vision of the project is (and probably the goals), and who keeps this in mind in approaching every decision that is made.  The leader is the person who cares the most about the project, and who seems to make the most cogent arguments about the project as a whole ("Which feature should we drop?", "How do we best fulfil the spec in this case?", and so forth), and to whom these questions tend to become asked of first.

This person doesn't have to be a project manager or the "lead" developer.  It could be a designer, for example, who feels excited by the prospect of delivering something worthwhile to the users of the project.  This person instills this excitement and/or caring, in some small way, to the rest of the team, and provides a degree of reward (it feels nice to make someone happy in addition to just doing your job).

Excitement and attachment, though, have their downside.  If the de facto leader is restricted from providing such leadership (even if it is only a perceived restriction) by the de jure management (for example, a project or line manager), problems start to appear that are hard to diagnose.  Things that "just happened" stop "just happening".  It becomes harder to make decisions or reach concensus about the project.  Things that were decided are forgotten, arguments about previous decisions start up again, and so forth.  Mostly, morale goes down.

My current experiment is trying to understand who holds the de facto leadership power in situations, and to understand my own role in things in terms of non-granted (ie, earned instead of granted) power - both in the past and going forward.  Even if I'm not right about defining the "project leader", this exercise has had some interesting results so far. 

Getting out of the never-ending mad dash that ruled my life for much of last year is starting to give me time to think about and hopefully learn from the events from that time.

Lately this thinking has revolved around how to do technology right, and how technology should be treated by business.  There's always lip service about what technology should be to a business, and how the engagements should work, and so forth.  But I think we so often find ourselves working from a flawed understanding of what success is to the business that is initiating the project.

A common position I end up in is taking over what pretty much anyone would call a failed project - a project where a lot of money has been spent and the resultant product is not even of sufficient quality to expose to others, let alone started to make money.  Aspects of the project that have already been paid for have been found not to be of sufficient quality (or even delivered at all), and need to have more money spent on them.

But I hack on the existing project for a few months to get it to the point where I wouldn't put a bag over my head to avoid being associated with it.  The project launches, and after a few rough patches and late nights, I get the last major kinks out, and the project starts making money.  Some might even call it a "success".

But is it?  If I'd been on the project from the beginning, my arrogance forces me to insist that the project would have been delivered properly the first time, with less back and forth to QA, with fewer publically-visible problems post-launch, and thus would be cheaper in terms of direct costs and indirect costs to regain the confidence lost during this whole process.  So, while the second part of the project (which some may call another project entirely) in itself is successful, the entire process of developing the technology was more expensive than it needed to be.

And that's not even the most damning difference.  If I'd been on the project from the beginning, my arrogance forces me to believe that the end result would be better in the way that most matters about technology in the long run - how it facilitates or hinders changes in future.  Or, put simply, a solution I'm involved in would give the company agility.

Many companies have a great idea, and get to market by the skin of the teeth of the people they could find and afford to do their initial technology.  And, because of the way the technology was rushed and was cobbled together, changing it is often scarily hard.  Which isn't a problem at first (when they have the existing developers who remember all the ins and outs of the system), and so the company's great idea and timing and efforts make them profitable and renowned.  Fast forward a few years, and they're cursing their main source of income and often the thing they're most famous for (if you're talking a web site, for example).

Why?

The developers are usually gone by now.  Even if they aren't, they no longer have the ability to keep in their heads all the hacks they've been forced to put in to the original system to keep it ticking over "until we rebuild it all".  There's entire pages of code dedicated to special cases for particular user names, groups, and so forth.  When changes do happen, all effort is put into reducing the intrusion on the original system, because the company has been bitten by the hard-to-detect consequences of previous changes.

Success or failure?

At the beginning, a great idea was all that was needed to get into the market, and delivery could be made timeously.  Now, there are more people available, and there's more money available.  Sounds good...  But, despite the additional people and money, even small ideas and small changes take a long time to make, and the next great idea is near impossible to implement.  These are the small and big ideas and changes necessary to continue being relevant - to not become an also-ran of the very area they once owned.

Unless your business usage of the technology is once-off - a gimmick web site that has a specified shelf life - then you have got to think about the cost of change built into that technology.  (By the way, this is also why open source and/or open standards are such a no-brainer in the long term - you never end up having your data and processes tied up in some proprietary system that makes the cost of change too high when you need to change something about your business.)

I don't claim to have the answer to how one can define success, but this is the simplest way to describe what I'm feeling now:

Success can't be measured only by what you have achieved - the resources, the accolades, and the good will.  More important than those, it should be measured also by what you can achieve from this point on.

(This also applies to any development project independently of the business that initiates it.  Taking the laziness as a virtue approach, you can measure your success by how much effort you save yourself by building something that's easy to maintain and extend in future.)

One of the constants in software development is dealing with people who just don't understand why software development doesn't run as smoothly as they expect - that the estimates are not met consistently, and that things that sound simple often aren't.  These are usually the people who, after promising not to change the spec because you explain how doing so will almost certainly cause the project to fail and the developers to leave, change it in a way that invalidates much of what has been done, introduces new areas of ambiguity, and expects it all to be delivered on the original estimate delivery date.

For them, I now have the following explanation about why software is hard (thanks to Outis in a letter in response to an article on Salon.com)

Programming is hard because that is generally the first point in the process where someone cannot hide behind unspecified assumptions, vague intentions, and requirements which may seem ideal as separate elements but turn out to be contradictory when combined

The programming stage introduces a level of honesty not enforced in earlier stages.  No longer can one hand-wave over how to achieve something, and there's nobody lower down the chain to delegate decisions to.

The most detrimental mistakes in the programming stage of a project are those that stem from ambiguity in the specification of the work to be done.  This is made even worse if the developers are not given the overarching vision and operating parameters of the project - so that they have a better than average chance to realise when an ambiguity actually arises - it'll just feel wrong.

If you've not managed to be honest in your previous stages in the development process, the programming stage is the most morbid phase for everyone.  For those involved in the earlier stages of a project, all they hear is that they've not done their job correctly.  (But, more realistically, I imagine they just think they have bad programmers.  It's amazing how easy it is to just blame it on the programmers for not "realising the vision" or not "knowing how to overcome technical difficulties" or just "being crybabies".)

For the programmers, all they hear is that fixing the mistake properly at this stage of the project will be too costly, and can't we please just use a hack just this one time?

The next most detrimental mistakes in the programming stage of a project stem from not hiring the best programmers to work on the project. Not hiring the best programmers will mean that you'll not be informed of the ambiguities and lacks of the specification.  Which is quite entertaining when they deliver something entirely unlike what you want.  And then you have to do it again.  And, well, they've not really got any better the second time around.

Or they'll cave a lot to putting in hacks, which is usually the most entertaining during maintenance and in future iterations of development on the project.

But then, I still see companies who don't care enough about their projects to have proper version control and bug trackers.

Maybe I've been living in the dark for ages and everyone else knew about it and didn't tell me, but Firebug is hella cool.  Firebug is a Firefox extension that makes web development life worth not committing suicide over - and version 1.0 has just been released.  It's like Web Developer Extension on steroids, although it's still useful to keep Web Developer around.

It has a Javascript debugger, which not only shows JS errors as you run, but allows you to pause execution with a breakpoint (optionally with conditions), and allows you to introspect values by hovering your mouse pointer over variables in the source code display.  You can also execute arbitrary Javascript in a Javascript console.  Oh, and watch your XMLHttpRequests as they occur.

In terms of HTML, it allows you to drill down to elements via the DOM or by using an inspector to choose the element you want by moving your mouse to the element.  It lists the full element path to the element, and all the styles that apply to the element, and allows you to edit those styles - even if in multiple different files all over the Internet.  You can also view and manipulate the style via the stylesheets without reference to a particular HTML element.

Oh, and you can graphically see exactly what files were requested for this page, when the requests started, and how long that request took to be fulfilled.  And the headers sent in the request, and the headers returned.  And the content.  Useful for your XMLHttpRequests...

Colour me impressed.