Being the 10th anniversary of when this story took place, I would like to tell a story today. It is one of the many stories that I tell each of my student mentees each year. At first it may seem like I am airing dirty laundry, but the purpose of this parable is that the reader can hopefully steer clear of the mistakes I made once upon a time.

I was working at the main IT department for UBC (then called ITServices, it is now called UBCIT) as a software developer. It was my first job after graduating from UBC (I previously held the job before graduation, but that's beside the point). I had been working at UBC for 2 years before a new project was started: Campus Wide Login (CWL). At the time, this project was revolutionary. It was a single sign-on (SSO) system that would obsolete the various usernames and passwords around the campus, and centralize it all into one system. The design was based on Kerberos which is a ticketing system, very similar to today's OAuth2 system.

The fact that this project centralized all authentication systems on campus made it a huge task, but it also suffered from a lack of developers. The project was split into two tasks: management and authentication (client) interface. The management interface was tackled first because all data needed to be organized and stored within an Oracle database, plus the various Web UIs needed to be created (a management UI, administrator UI, guest UI, and end-user UI). I was the sole developer and principle architect on the project, but I also had a database administrator and the project lead (for the vision stuff) on hand. This was my first mistake: I was young and inpetuous but I didn't realize that having one person as the developer and architect leads to an echo chamber, not to mention a lot of stress. I recall fondly (okay, angrily) when the project lead gave me a 2 week deadline for the entire system to be up and running. Spoiler alert: the project took 1.5 years.

As mentioned, I was young and since this was my first real job, I was out to impress everyone with my l33t skillz. I created the database design first and had it reviewed by the DBA for errors. I then designed and created the management code, which relied heavily on XML-RPC as this was going to be a Web Services project (many, many years before that name became overused). As the sole developer it was up to me to prevent others from scope creeping and trying to make the system more difficult1 than it was supposed to be, nevertheless it still managed to become bloated. I shudder to think about how many anti-patterns I may have created. I know I sinned greatly by creating a God class for the kernel of the system.

Okay, let's fast forward as the development progresses and I start creating the administrator, management and end-user Web UIs. I was also tasked with creating the authentication system because the original system was taking too long to implement (and even then, it would never work 2). About 8 months into development I get some help by way of a junior developer, however this is a double-edged sword. He was so junior that I needed to teach him Java, programming techniques, and other fundamentals which meant I am now context switching even more throughout the day. As it turns out, my brain was not up to the task and I always had more work at the end of the workday than I expected. I started taking work home so that I could get the work finished on time. This is mistake #2. Of course, when you are so far in the thick of it, you (or rather, your brain) rationalizes it away.

You can only rationalize it for so long. Eventually cognitive dissonance starts to creep into your consciousness. Now CD is a difficult thing to see coming. It is there to keep you from doing things that you rightly know are bad for you; however, if you continue to disobey your psyche, your psyche starts to change and accept the bad behaviour. I started getting moody, achy, and nervous about any kind of failure. I was always in a high-stress situation in my mind because I needed to get so much work done. At this point, you are asking yourself, why didn't I just push back? That's something that you learn after years of work, but when you are young and impressionable you will work yourself to death just because you don't know any better.

The system was almost live and I was busy trying to get everything ready. I was letting my perfectionism get a complete hold of me and not allowing anything to go out to QA because it just wasn't perfect enough. Not until my co-worker, Zed Shaw, told me that the product was good enough3 that I let go and we deployed it. The system worked fairly well and I was quite happy, but I just couldn't lower my stress level, after having it so high for so long. That is mistake #3: I should have opened up to upper management and spoken to others about how much stress I was under. I tried a half-assed attempt at explaining the situation a few times, but I was also filled with hubris that I would be showing I'm not some l33t hacker and was susceptible to human failings (there's mistake #4 for those keeping track at home).

The success of the system caught the eye of some people on campus. One fine late summer day I was asked to go to lunch with my Director and Manager, as well as a Dean of one of UBC's departments. I was quite impressed that anyone would want to talk with me and pick my brain, so I went along. What actually happened was anything but praise for my work. The Dean told me that I had done wonderful work and that he had an opportunity for me. His idea was to take over the CWL project and create an "elite" (his words) team consisting of himself as the project lead, myself as the developer, and his son as the other developer. I quickly saw this for the nepotism that it was, but something else bothered me more. I had a junior developer (and one other developer, by the time the system was launched) that reported to me. I would have to kick them off of the team, through no fault of their own, just so that I could step one rung higher on the corporate ladder. My ethics just couldn't handle that and I said no. Now, I'm not sure if Deans don't like to be told no, or if there was something about my age, or what-have-you, but he went off on me. In a public place, berating me while my Director and my Manager watched. At one point I even openly asked for help from my Director and Manager to come to my aid, because it was all to much for me.

Remember how I had allowed myself to become high-strung because of my need to complete the project? Well, receiving a public berating absolutely crushes someone's psyche. I could feel myself welling up at the table as the other three left. I sat there for a long time trying not to cry, but I couldn't understand why I felt this way (other than the obvious, of course). This was mistake #5, to allow myself to get too emotionally invested in a project. I still fully stand by my decision to not screw over the people that report to me, but there's a bit of mental scar tissue from that lunch.

I lurched back to work and over the next month I really attempted to get myself working again. I didn't realize it at the moment but I had completely burned myself out on the project and the fault rested solely with me. Of course, although CWL was in Production it still needed maintenance and development tasks to be completed, but I had lost the passion. Within 3 weeks I would tender my resignation and leave UBC. I felt terrible, and I completely blamed myself for a very long time 4. I pushed myself too hard, too fast and abused just about every piece of me just to gain the admiration of those around me, when really I deserved no admiration at all.

At this point you are thinking that I'm being too harsh on myself, not deserving any admiration. However, that's not the point. When I've managed other people, I am quick to highlight the successes as well as the failures, but but I don't go on and on about either of them. I feel as though I was expecting some ticker-tape parade for the extra hours that I spent, of my own free will, when really I should have been trying to maintain a proper work-life balance. In summary, I didn't deserve admiration because I had completely wrecked myself and burned out just to meet some crazy goal that could have been reached if I had calmly worked on.

The silver lining to all of this is that CWL is still up and running today, serving well over 50,000 people daily (during school term). Almost all of the code running in Production is still my original code, and it requires very little RAM and CPU (mainly just horizontal scaling due to concurrent connections). I call that a significant achievement and a result that would make any perfectionist proud.

This is the story that I tell my mentees because I believe it is important to realize the signs and symptoms of burnout. We as software developers use our brains more than anything else, and to abuse it can be very detrimental to a promising career. It took me a year to get over the burnout and I still find myself riding the knife's edge between focus and burnout 5, but I have learned to listen to those around me when I cannot see it myself.

One last note: I worked with some really great and famous people during my time at UBC. I would like to take this time to very belatedly thank them for their help, talent, advice and support.

  1. One particular example of this was my project lead spending 2 hours of a meeting trying to get a consensus on whether we should make the first name and last name fields in the database required (i.e. not null) or not, because some people may come from different cultures and have only one name (e.g. Cher). 

  2. The main reason that the authentication system would never work in Production is because the designer (and developer, he was the same person) didn't understand how the Web truly worked. This is a disaster and why I truly believe that you can't just BS your way into building big systems. Even if you aren't found out, your code will highlight your mistakes. In this case, I had a very heated and lively argument with the designer because he was trying to tell me that the Web was stateful, when in fact it is not. His authentication code would have you create a connection (akin to a TCP connection), do your work, and then disconnect. He vehemently disagreed with me when I told him that each of those 3 tasks would be 3 disparate calls due to the statelessness of IP and HTTP. Eventually the error was found and I was put in charge of co-designing the authentication system with the wonderful Avi Bryant, who then implemented it. 

  3. Although I have never told him, I considered him my mentor. Aww shucks. Thanks Zed. And to the Internet, his bark truly is worse than his bite. Don't pay any mind to what the rumours say about him. 

  4. Okay, to be honest I still blame myself a little bit to this day. But I try to use that blame to keep my eyes open to the signs of burnout. 

  5. There's a lot of stories yet to tell of me coming close to burnout. I have a feeling it is due to my perfectionist tendencies, as well as a need to please everyone around me. But that's a story for another time. 

« Previous: Setting up a VPS
Next: Signs of Burnout »