Are You Really Looking for a SENIOR Software Engineer?

Over the past four months or so, I have been in the job market after being laid off from my last gig. I have been proudly sporting the #OPENTOWORK hashtag on LinkedIn and, of course, applying for every possible position for which I am a reasonable fit. It has been a process of “kissing a lot of frogs” (see Quote Investigator for what this means) and I have become increasingly frustrated by the process.

Apart from the familiar frustrations of ghosting, job postings that turn out to be more fiction than fact, and so on, I have discovered that everyone wants to hire “Senior” Software Engineers. Yet, has this now become just another buzzword in the industry? Do prospective employers who need coders or junior-level developers just automatically jump to referring to the open position as a need for a Senior Software Engineer?

There are a few reasons that I look sideways at many of these positions. The first and most obvious indicator is the pay scale. There is a range for Senior Software Engineers that makes sense and which can be discovered within minutes of researching and yet I have seen entry-level ranges applied to some of these offerings! So, let’s scrap any of these and continue looking through the requirements. So many of them read like entry-level positions. For example, in one I recently saw, there was a requirement that the candidate “must know about version control such as Git or SVN.” Really? That is like asking for a professional Lead Guitarist from a well-known group and then in the interview, asking them to play a G chord for you!

While we are on the subject of requirements, so many of these harp on skills that should be easy for a true Senior Software Engineer to pick up quickly. It makes me wonder if the desire of the employer is to quickly backfill a position after someone moved on. In other words, they are looking more for a replaceable puzzle piece than for a solid contributor who can move their project(s) forward. How many of them require a specific language, for example? Again, how many Senior Software Engineers can only function in one language? Most of us, if not all, are polyglots in terms of development languages and scripting dialects! Heck, in my arsenal is Java, C#, Typescript, Kotlin, Javascript, Python, 80×86 assembly, C++, C and those are certainly not the only ones by a long shot. How about the ones we dabble with “just enough to make us dangerous?” In my case, Rust, Go, and so on. (Oh, and did I mention Pascal? That really makes me a Senior…) I certainly resonate with Dave Farley who states that “our job is to solve problems for people, largely using software” (see, for example, this Dave Farley short) and not to just write code.

Finally, assuming that a candidate makes it through the hurdles and gets through to a battery of interviews, how many of these interviews for a Senior Software Engineer include creating a “Hello World” type of application? Now, don’t get me wrong…I love doing katas and belong to CodeWars and Exercism. These exercises are great to help a developer to think clearly, hone approaches over time, and sometimes how to reason out-of-the-box solutions. But in the stress of a 30-60 minute interview, under scrutiny by a stranger, and using tools which are unfamiliar, what exactly does this prove other than you can figure out how to deal with a variation of FizzBuzz, Game of Life, or Gilded Rose? If the need is to know how a candidate codes and/or how they think while coding, would it make more sense to spend some quality time looking through some of their GitHub projects? Heck, how about interactively going through some of the candidate’s projects in their own codebase during one of the interview sessions? Shouldn’t these interviews be in-depth back-and-forth discussions about solving problems instead? After all, this IS the reason one should be seeking a Senior level engineer.

The key is to determine what actually makes a software engineer a “Senior.” Throughout the history of mankind’s professional endeavors, there has always existed a gradient in expertise. We begin a career as raw material which is forged and built up through experience until more and more expertise is accumulated. To become a “Senior” at anything, time is involved, of course, but it is more than a matter of time. It is a matter of application of acquired knowledge and, in many cases, avoidance of “youthful” exuberance such as reacting to situations or accepting the first idea that seems to be right. Personally, I really resonate with the work of the Dreyfus brothers at UC as they examined the nursing profession (see The Dreyfus Model of Skill Acquisition). Everyone’s experience exists in a continuum between Novice and Expert in every aspect of their pursuits and this includes software engineering for us engineers, of course.

So, the question of recruiting should be what does a Senior Software Developer bring to the table? I believe that there are 6-“R”s that define the envelope surrounding this position as outlined below.

Recognition – Senior level engineers have experienced a lot of things, both personally and through the lore of others, through their many years on the job. Because of this, they can recognize pitfalls, problems, and even positive decisions quicker than most. It is this recognition ability that helps them steer a project towards patterns or away from anti-patterns in the realms of coding, architecture, infrastructure, and project execution.

Reading – Senior level engineers who are good practitioners of their art constantly strive to keep their skillset sharp. They read voraciously, train, attend conferences, pick up pet projects that challenge them to learn new tricks and approaches to problem solving. This serves a company kind of like the luggage in the cargo compartment of an airplane. It is there when it is needed and even those things that seemed to have been silly to pack in the first place sometimes end up saving the day!

Reflection – Senior Software Engineer is more than a title. It is a way of life for those who have really earned that title. They are engineers, thinkers who delight in being tasked with difficult problems to solve. The journey towards seniority has taught many lessons in choosing wisely while solving problems and solid senior engineers take this seriously. Reflection permits them to sidestep decisions which have led to disaster in the past and, instead, leads to providing solutions that are adequate (not overengineered) and maintainable.

Research – Another benefit to years and decades of experience that many Senior level engineers exhibit is that they don’t just blindly accept the quick answers. Many have cut their teeth through the failures of frameworks (those that promised the world but only delivered a small island) and vendor-hype. They will take their time to determine if a solution truly can be the right one by doing their homework. They will also spend time on determining how solutions can be combined in powerful and effective ways to accomplish the goals of the project without compromising its viability and maintainability.

Reason or Rationalization – Excitability is an important trait because it empowers the dreamers and forward-thinking innovators to make new things come about. Yet, solid and maintainable software cannot be developed simply by hair-trigger excitement alone. Senior level software engineers have learned how to use reason to balance the dream with the execution. For every Steve Jobs with their incredible vision, there needs to be a Steve Wozniak who can anchor that vision in reality. Together, both sides of the equation come together and produce something much greater than the sum of its parts. Senior level engineers’ experience drive the stakes of practicality into the ground to form the foundation for the vision to thrive.

Reciprocality – Finally, most Senior level software engineers have learned that there is a tremendous amount of power in collaborative thinking. The sum of two honed minds is much larger than two, and their experience has led them to understand that taking the “lone wolf” approach to software engineering is not often the best way. They seek to bounce ideas back and forth among the team and, because they are aware of the relentless march of time in their own lives, they also work to train up the junior developers and engineers to one day take the reins.

So, to return to the premise of this article, the question is “are you looking for a SENIOR software engineer?” If so, then hopefully you are willing to have the right type of conversations to determine if the candidate is indeed qualified. That person is more than a resume, more than a coder, more than someone who can spout (mostly meaningless, in a day-to-day sense) information that can be found in less than a second’s internet search. Are many or any of the 6-“R”s part of your actual job need and, if so, are they in your requirements and do they play an important part of your interview process? After all, you are looking to hire brain-trust to guide your project and, maybe even, the hopes of your company into the future!

Church Body Democracy (or Not)?

I am finally getting some words down on a subject that I have prayerfully mused over for quite a while. Recently several events have brought it to a point and have forced it to the center of my thoughts and thus it has to be worked out. As with all my musings, this is not binding in any way except where Scripture clearly states thus-and-such. Also, it is my hope that it will serve to ferment your own processing of this subject and not serve as a platform for battle. (Should anyone want to make this a battleground, let me plainly state that I will not engage with you. There are hills that I am willing to fight the good fight and even die on, but this will not become one of them.)

Before jumping into the meat of the article, I also commend an article I wrote 9 years ago entitled How to Reform the Church. It may also prove to be of tangential interest to this discussion.

Beginning with some context and history, I am specifically dealing with the democratic membership of Southern Baptist churches and my understanding is that the root of this specific rule within the congregation is similar to that of many Protestant bodies. The fear that it addressed was the possibility that the congregations of churches might be bound by bad decisions of church leaders that would lead them into bondage similar to that of the “papist” church culture that Protestant leaders fought against, and many had even spilled their blood to secure. However, I believe that this power has been take completely out of its original (and well-meaning) context and corrupted horribly over the years.

As with all things concerning God, salvation, sin, and church subjects there is no room for any rule or practice that violates God’s Word. And, it must be said that God has a lot to say when it comes to the Body of Christ, the Bride of Christ, which is the church. Also, to be very clear, God does not state that the ruling of the Body lies in the pews and, even though He has assigned leadership for the Body in the offices of Elders and Deacons, their rule is also subject to Christ and to the Word. Violation of God’s authority in any matter, including the existence and practices of each instance of the Body, is clearly sinful. Therefore, anything that is added to church life must concur with Christ’s lordship over His Body.

Last year, while filling the pulpit one Sunday for my pastor, I preached (at his suggestion) a message entitled “Principles for an Enduring Congregation.” This was built upon some of the principles in 1 Corinthians 1-4 concerning problems that the church at Corinth had experienced. More recently, I was led to preach a message entitled “Pieces, Parts, and Wholeness” (available for now as a Message on YouTube) that expounded 1 Corinthians 12. Not only this, but I had the pleasure of reading “Devoted to God’s Church” by Sinclair Ferguson (highly recommended, if you have not had the opportunity to read it yet) and I also had the opportunity to hear Paul Washer’s original unabridged message “10 Indictments Against the Modern Church.” Finally, my pastor and I have had a fair amount of back-and-forth discussion concerning this very subject as we found ourselves confronting this over the last umpteen months as we shared our Christian walk.

Before I deal directly with the subject of church democracy, let me first outline a collection of my current understandings and positions concerning the subject based upon all of these sources and others.

I believe in a properly formed local church body that is (of course) under Christ’s shepherding. The key to this shepherding as He assigned it is outlined in 1 Peter 5:1-4. He has assigned one or more elders to shepherd “God’s flock under (their) care, watching over them” as God wants them to be willing to do. It is not clear in this passage if the elders are associated with one church (a “plurality of elders”) or are single pastors over each of many churches or some combination thereof. Since this epistle is not to a specific person (as is 1 Timothy or Titus) or one church (as in 1 Corinthians), Peter may be, and probably is, addressing “all of you elders/pastors in all of your church bodies.” It is clear then that there is an authority gradient that starts at the top with Jesus Christ and which extends to the eldership of the local church.

It is imperative to head off the thought that the elder or pastor is somehow elevated above the flock before Christ, as some people do. Each church leader is just as much a sinner saved by grace as any of their flock and are not somehow made into supermen or half-gods! With this having been said, they have been extended some authority from Jesus in order to accomplish the task that they have been called to do. They are to shepherd (that is to feed, lead and protect) the flock that they Jesus has placed in their care. And they are to do so (verse 3) “not lording it over those entrusted to (them) but by being examples to the flock!” The authority that they have been given is that which Jesus modeled before the disciples on the night He was betrayed, a lesson that Peter certainly never forgot (John 13:1-17). It is a life of leading by serving, humbling by being humble. The motivation of the vocation to be an elder is not money (verse 2), nor compulsion, but instead must be a calling to serve willingly.

I believe, as many do, that a church should be administered by a plurality of elders. Even small churches with congregations of 25-50 members should aim at having 3 elders. This having been said, we must be careful in assigning elders just to reach some numerical goal! The principle of “do not be hasty in the laying on of hands” (1 Timothy 5:22) must apply here as anywhere. Remember that elders are to be called, not made. As God raises up leaders in the midst of a church and they prove themselves to be faithful servants, then they should be prayerfully considered for eldership. This is especially true when you consider the roles elders need to fulfill as they lead the church. Titus 1:6-9 states that an elder (overseer in some translations) “manages God’s household,” faithful to his wife (and I take this more than just in marital faithfulness, but in the terms of Ephesians 5:25-33 of loving his wife as Christ loves the church), must not be overbearing, exhibiting hospitable tendencies, loving what is good, living a self-controlled and holy life at all times, and must be completely unswerving in the primacy and supremacy of God’s Word.

Regardless of if a church has a plurality of elders or not (or even agrees that there should be a plurality of elders), there is always one elder who is required, the pastor. The pastor must meet all of the criteria for eldership discussed above from Titus 1 as well as in 1 Timothy 3:1-7 and (this is important) must continue meeting the criteria for the duration of their service! Over the past decades, the amount of misbehavior that has been exhibited in the lives of some pastors has served to bring shame to the name of Christ, hasn’t it? Pardon the digression…

In addition to the office of elder we have the office of deacon. Deacons are responsible for the physical care and feeding of the church. This church role was established in Acts 6:1-7 and serves to permit the elders to focus on “the ministry of the word of God.” It should be clear that deacons are not given authority over the flock nor their role is not intended as a teaching ministry akin to that of the elders. Instead, they are chosen to serve their local church body. Unfortunately, in many congregations the role of deacon has been (incorrectly) elevated to an eldership, one of administrative and spiritual leadership, instead of one of tender care for the needs within their flock. This tends to lead to an incorrect and invalid type of leadership within Christ’s local body.

It is essential at this point to affirm that any church which is Christ’s is indeed His and uncompromisingly be subject to His complete rule. Any group, no matter how kindly or wise it may be, that does not subject itself to His total lordship is not a church. At no level within the Body of Christ must anything be introduced that contradicts God’s Word or His perfect law for (again) it is all His. The eldership, the deacons, or the flock are not permitted to decide to follow their sentiments, their feelings, the way of the world, or anything else that usurps Jesus’ authority. Likewise, there is no democratic voting process or despotic proclamation that can establish any such contradiction. Grasping this should form our initial understanding that the church is not a democracy, at least in terms of defying the living God.

Thus, the basic rule is that church is never to be subject to arbitrary whims. It is right in the middle of this rule where the majority of congregations are going off the rails today. There is not an implied nor a conferred right to convert any of God’s full-stops into commas and to append man-made clauses or erasures to His will! He doesn’t change His mind or sacrifice His righteousness concerning sin or salvation.

So now we have broached the premise of the title. Beware of any elder, deacon, or member of the church who changes the mission away from Christ’s commission, who changes the ownership of the church body from Jesus, and/or changes God’s framework of righteousness to anything but. This is not to say that the church is not a place where sinners come to discover the freeing Word, but it is a place that will not absorb or conform to the life of sin. To put it in modern terms, you should be able to come to the church as you are, but be assured that you will not remain as you are! The gospel must have its effect on lives that need to be brought to the Holy Spirit’s conviction. But the church is never going to change to accommodate sin because it does not measure up to man’s whims. Of course, this does not prevent congregations from doing this. Sadly, what they don’t realize is that, as soon as they usurp Christ’s rule over His body (using some sort of democratic process), their congregation begins a slide downhill that ultimately separates them from the rest of His legitimate church, the Body.

It is sad to observe the operation of many congregations today insofar as how they use their democracy to spit in Christ’s face. This takes several forms, such as:

• Members who rebel against proper (Biblically correct) leadership administered by the pastor/eldership;
• Members who rebel against properly handled and administered church discipline;
• Members who nominate and vote-in deacons and elders who fail Biblical qualifications;
• Members who move to override scriptural truths either by establishing rules in the church’s by-laws or through committees;
• Elders or deacons who attempt to exert “authority” and establish practices and rules that are against God’s Word;
• Deacons who rise up against a Biblically correct pastor and subsume the role of leadership; and
• Elders or deacons who somehow place themselves into Christ’s leadership role.

These are all egregious overreaches of the basic church “democracy” that was introduced following the Reformation. The principle reason for this was mentioned earlier, namely to prevent a “papist” approach in the church movements of the Reformation, which would manifest itself as man-made dictates usurping Christ’s lordship and the Word of God. This power was never intended to be a tool that could be whipped out and used to batter a congregation or leadership into a bloody pulp! What is absolutely important is that this democratic behavior is not dictated in the Bible at all. We actually see the God-assigned authority of the eldership executed instead.

So, what do we need to do with this democracy then? We must treat it as a trust and not some sort of right that we, the flock, flaunt willy-nilly. We must keep it tucked away behind a glass panel clearly marked “Break glass only in case of apostasy!” The only proper use of this is to prevent the church from being hijacked by falsehood and all use of it must be very carefully, as lovingly as possible, and prayerfully applied by a church body that is well-versed in Scripture.

The closest analogy to this is the reaction of the brave passengers and crew on United’s flight 93 during the horrible events that unfolded on 9/11. They came to understand that the hijackers were planning to use their flight as a bomb as other planes had been. It became a fight to the death situation for them as they enforced their consensus that “No, you will not accomplish your nefarious plans” in the faces of the hijackers. This is the only proper use of the democratic trust process within the church. It serves as a last line of defense for the body and prevents the church from being yanked out of Christ’s hands and misused.

In closing, I know that I barely scratched the surface of this important subject. I also did not extensively provide Biblical references to keep the article as readable as possible. If you want me to add these, drop me a comment and I will respond. Hopefully, either way, it will actually get you digging your own references up.

Burdens of an (Introverted) Solo Practitioner

While I may be alone in my pursuits, I know that I am not alone in my state of anguish.  Surely, of all of the billions of people in this world, there must be others out there who struggle with my same angst, namely on forging forward in practicing the activities of our professional lives devoid of a supporting community of others. This blog post is different from my usual ones because it is, admittedly, more personal.  Please bear with me as I peel back just a bit of my outside shell and share something deeper.  It is really, in the end, not about me but about the myriad of other “me’s” out there in the world who suffer in roughly the same way.

By way of introduction, I have two professional pursuits.  Here the term “professional” means that they demand copious amounts of time and study to be accomplished and that the results of these pursuits are under scrutiny and evaluation by others.  Just for the sake of full disclosure, week-in and week-out, I am both a software engineer and a lay preacher. Apart from mentioning them here, these are inconsequential to this discussion.  I face the same burden in either of these spheres as do others like me who are doctors, civil engineers, nuclear physicists, stock analysts, nurses, teachers, and so on.

There are those who are natural extroverts, who are gregarious characters that most people will describe as being “larger than life!”  You are the folks who always gravitate to the center of a room and never meet a stranger.  You always lead with a handshake and an introduction, and you are never timid in any situation. If you are one of these characters, you are most blessed.  In that same room are one or more people who seek the walls and corners and try not to be noticed, successfully so most of the time.  We are the introverted, those who are uncertain of ourselves, those who have a lot to give but cannot escape our shells long enough to get more than arms reach away from that wall or corner.

Many of us have become “functional extroverts” over time by necessity because our professional callings have required it.  There are moments in our days in which if someone took a photo of us in action, it would display us shaking hands, talking one on one with others while looking them squarely in the eye, leading meetings, and so on.  Were you to rely on those snapshots, you may be led to think that we ARE extroverted and completely comfortable in who we are.  Yet, if those snapshots were actually X-rays, you would see that we each hide a chunk of a wall or comfortable corner inside and are hugging it with all of our strength.  We are still alone, desperately alone, in our respective professional worlds.

Our greatest desire is to become a part of those who share our same pursuits.  Personally, I agonize to be a part of a group of software engineers or a part of a group of preachers, not as an outsider hovering around the edges like a stray pariah, but as part of the dynamic center.  Does this mean that I don’t have friends in both of my spheres of professional interest?  No, but it means that these friendships never rise to the ongoing vibrant collaborative fraternity that I desire and need. 

I know that this must be the same for many others out there.  We crave this vital connection with our peers that we see others enjoy, but somehow, we never can connect to it.  Our souls cry out (silently, of course), “work with me; share with me; collaborate with me; could you just be my friend in our shared professional lives!”  We never seem to be able to “belong” and this is not from a lack of trying.  We try, we bare our very souls (painfully) sometimes, we show ourselves as faithful friends, and nothing changes.  We remain solo practitioners of our respective crafts.

In my own life, I did experience this blissful collaborative friendship, which probably speaks more to the graciousness and kindheartedness of my friends than for anything in my ability.  While working as President at Netpath (a small Internet company founded in 1994), for many years my friends, Jeff (who was our VP) and Jesse (who was our Director of Operations) and I had a standing Friday night gathering where we would sit around my kitchen table with our laptops and a bag of chips (ALWAYS Wise Ridgies, for that was Jeff’s preferred) and toyed with code, debated software topics, and so on.  I knew it then, and definitely know it now, that those moments did more to grow me as a software engineer than most others.  Of course, like all good things, they came to an end.  Jesse had to move on, and Jeff fought an undiagnosable illness and died unexpectedly.  But I digress but the point is that I KNOW that what I crave is possible.

So, the question remains on how to deal with this angst and get a handle on this burden?  What is the formula?  Is the answer built on just keeping on trying and trying to get where we want and need to be?  In other words, should we be like the donkey who desired to be a racehorse in the story (see https://steemit.com/hive-122108/@giantbear/the-little-donkey-that-wanted-to-be-a-race-horse-an-original-story-for-children-fiction)?  Or is the answer just to let it go and just “be the best you that you can be” in spite of all?  Or maybe there a set of 10 magic exercises that will solve the issues that seem to prevent me from achieving this sense of belonging?  What is the answer?

Don’t “Go Your Own Way”

(With apologies to Fleetwood Mac for the title of this piece. It will make sense later, I promise)

Today’s article is once again taken from the annals of disasters as was the previous installment (Poor Engineering Decision-making) and was influenced by another engineering-disaster show I caught a few weeks ago. In the series “Deadly Engineering” on the Science Channel, one of the episodes outlines two train collisions, one during the 1980s in England and the other 30 years later in Turkey.

In the English rail system in use during 1988, a failsafe system was used to prohibit multiple trains from being on the same section of track. It functioned by a train on the track within a section shorting an electrical circuit which then activated the signaling system.

In the Clapham Junction disaster which occurred on December 12, 1988, the root cause was that the system intended to prevent collisions malfunctioned horribly. A full train on its way to London entered a section of track which was signaling that it was all clear (green) and plowed into another train that had stopped for a red signal on the same track. Trains were derailed and one sideswiped another empty train on an adjacent track. When all was sorted out, 35 souls were killed and 484 were injured.

Clearing up after the December 1988 crash west of Clapham Junction (Wikimedia Commons)

What caused this failsafe system to fail with such tragic consequences? The outcome of the investigation into the incident will lead us into understanding the title of this article.

In the months leading up to the accident, work was being conducted on rewiring signals and upgrading the signal lights themselves. An electrical technician had been working on the signals on that section of track earlier and had rewired the signal unit. However, he had neglected to cut off the tails of the original wiring and these were left dangling even though they still had power. One of them eventually made contact within the control box in such a way that the signal on the corresponding section of track showed a false green.

During the official investigation of the signalman’s involvement in the sequence of events, it was discovered that he and his coworkers had been working incredible amounts of overtime and that there were systemic failures in all of the processes. In addition to the project workers being overworked and tired, it was also discovered that there was little or no supervision and enforcement of procedures such as counting wires (which would have exposed the wrong wire count in this case). Even more appalling, a nasty finding was exposed that over the decades leading up to the Clapham Junction disaster, signal technicians had grown slack and many had developed their own testing methods to shortcut standard testing processes!

Those of us in the software engineering world should take this history lesson to heart. This is especially true of those who work in the areas of our industry which touch on safety technologies. Of course, even those of us who work on (seemingly) innocuous open source projects may be oblivious to usages of our projects in critical applications!

There are two important lessons here that I want to call out. The first concerns the proper treatment of our brains by not pushing ourselves so hard that we do not get adequate amounts of rest between shifts. If we allow ourselves to be pushed so hard that we work ridiculous amounts of overtime for prolonged periods, something is bound to break somewhere in the cruddy code that we will begin producing. Longer term, exhaustion contributes to burnout. I don’t know about you but I have met my fair share of burned-out developers and had to deal with their negativity, their especially bad code, and their general lack of get-up-and-go. Frankly, I don’t ever want to be like that….ever.

However, the second point based upon the findings concerning the testing methods and processes is most important. Our industry has a number of well-established testing practices such as TDD, BDD, Integration Testing, CI automated testing, and so on. Each of these testing frameworks have well established rules for engaging them yet so often developers and testers try to “roll their own” implementations. How many times have you heard a developer say (or, maybe if we are honest to ourselves, we will admit we may have done this too) that, “I do TDD on my code,” and yet what they actually do is write unit tests after the fact? As most of us know, TDD follows a strict Red-Green-Refactor procedure to develop new code and not to wire existing code to some sort of happy test we cook up in our minds.

Things get extremely interesting when mocking libraries get thrown into the testing mix. These should be used sparingly to solve dependency problems but how often do we find tests that boil down to mocks testing other mocks brokered by minimal pieces of user code? When the state of testing is evaluated, many times our development and QA teams really are throwing out sound and established testing methods and either short-cutting them or creating ineffective replacements. Time pressures, poor project management processes, and proclamations from naive and non-technical management on high, rarely help the situation.

My hope is that this will serve to challenge us to not attempt to “go your own way” and work towards making our software as solid as possible.

Poor Engineering Decision-Making

I am a disaster-documentary fanatic, not because I am ghoulish and delight in tragedy, but because I consider myself an engineering sort of person and thus desire to learn all I can from others’ work. I chew up episodes of Mayday/Air Disasters, for example.  I also am sucked in by shows such as Seconds from Disaster (now no longer being produced) and recently came across a series on Discovery, When Big Things Go Wrong.  In Season 1, Episode 3 of this series, one of the linked investigations concerned the derailment of a high-speed Deutsche Bahn trainset at Eschede in Germany on June 3, 1998 which led to tremendous loss of life and limb. 

Eschede train disaster. This image was taken with a small hand held photo camera and shows the severe destruction of the rear passenger cars that were pushed into each other and into a road bridge. (Nils Fretwurst, Wikimedia Commons)

The story of the Eschede derailment (see https://en.wikipedia.org/wiki/Eschede_derailment) holds a number of lessons that engineers need to heed.  The one key lesson that I want to extract and apply to us software engineers is that we must be very careful about repurposing seemingly usable components from other use-cases no matter how similar they may be to our use-case.  The entire scenario that unfolded to become the worst high-speed rail disaster in the world was triggered by a single fatigue crack on a single wheel which, using perfect 20/20 engineering hindsight, should never have been used on that specific train!

When these high-speed ICE1 high-speed trains were originally designed, they were originally equipped with standard solid wheels. Unfortunately, at least in the scope of our story, these solid single-cast wheels could develop out-of-round and other conditions which would disturb their perfect balance at speed and thus lead to noticeable vibrations creeping into the passenger areas. Engineers were tasked with solving this problem.

The solution seemed to be in replacing the solid wheels with wheels that isolated the rolling surface from the hub using some sort of rubber ring. This was not, in itself, a new design since these “resilent wheels” were in use by several tram systems worldwide specifically to reduce noise and vibration. This design is sometimes referred to as a “steel tire” since a steel band is forced over a thick rubber band attached to the hub center as can be seen in the diagram below:

Sample modern resilent wheel design and components (image by Group Lucchini)

It is important to understand that this design was well tested within the domain it was being used, that is, for trams and other low-speed applications.

So, ignoring the oversimplification of thought-process, ultimately engineers decided that silent wheels for trams and other rail rolling-stock should fit the need for a quiet wheel on the ICE 1 trains. This is the danger of following similarity in use-cases in distinct domains. It is extremely important to ensure that such decisions really are valid because similar is not equivalent to sameness. In our software world, we are faced with these sorts of decisions all the time. There is a plethora of frameworks and infinite numbers of open-source and boxed solutions that constantly vye for our attention. These solutions present dozens of use-case scenarios that seem to match our specific design needs and are absolutely seductive choices to help us eliminate tons of research time and work.

The note of warning is I hope we derive is that these decisions may have safety, performance, and development consequences that have to be fully vetted and risks understood and weighed. This leads us into the rest of the sad Eschede story. Once the engineers had reasoned that these resilent wheels should meet their requirements for quiet operation, they pushed forward with designing them for the ICE 1 application.

One of the great mistakes that they made was not consulting other SMEs (Subject Matter Experts) in the industry. Japan had been working with high-speed trains since the early 1960s and France had kicked off their TGV train services in 1981. Other surrounding countries were also working on high-speed options. Thus, there was a considerable pool of talent and prior-art to tap for information. Another mistake that they eventually made was one which runs rampant in our software industry. Yes, there was a lack of proper and complete testing of the design. There were a significant lack of full high-speed testing of the new design to understand its limits and possible problem areas. This oversight was because there were no German facilities existing that could actually physically test the design at speeds approaching their operational limits. Thus, testing was mostly conducted with the low speeds and the engineers made qualified assumptions based on theory and understandings of the materials and pressures involved.

The new wheels accomplished their immediate goal of eliminating the dreaded vibrations so, if it works, as we all know, we can ship it! Another notable point is that the new wheels worked for many years without any failures. In our parallel universe, how often are we software architects and developers lulled into a sense of safety by years of flawless performance? We must always have an expectation of catastrophic failure triggered by a confluence of issues that all contribute to something unforeseen occurring.

The final mistake that was made and that is worth highlighting here is that of management and engineers not keeping a finger on the pulse of the industry. Üstra, the company that operated the low-speed tram services in Hanover, discovered that their resilent wheels were developing stress fractures running at the trams’ maximum speed of about 14-15 mph. They reported their findings to the industry at large and to DB directly. Had something been done about this finding immediately it is probable that the fatal event would not have ever occurred. The recourse, an expensive one admittedly, would have been to pull wheels from the ICE 1 trains, remove the steel tires and properly inspect them for stress and fatigue fractures. We all face this kind of decision based upon design choices we may have made years before. Either we appropriate the consequences of the choices or we will sweep them under the rug and blame others. As engineers, we need to be as sure of our original design choices as possible and understand that the day may come when we need to redesign or rebuild some part of that design based on new information coming to light.

Well, the rest of the story is documented in reports and documentaries. About a year after Üstra warned DB about the problems with their wheels, ICE 1 #51 had one of its steel tires on the first car fail catastrophically. It penetrated the floor of the car and dangled down to the point that it pulled a guide rail for a switching point away from the rails. This guide rail projected upwards and also penetrated the floor of the car, lifted the bogie containing the axles involved, and they ended up on the parallel track. This led to the track of the train being disturbed and the cars were violently thrown off their tracks at greater than 120 mph. Many travelers lost their lives in the subsequent melee.

We must be extremely deliberate in our design decisions. Software is not unlike hardware like these wheels in that it has tremendous power to cause chain reactions that may lead to loss of life. How do we know that the code we produce today will not be used to drive a self-driving vehicle, control safety equipment, support medical applications, keep aircraft in the air, or some other critical function. It is easy to point fingers at those who participated in the decisions that led to Eschede but are we fundamentally different from them in our own engineering capacities? Food for thought.

Keeping Notebooks in a Digital Age?

My current workplace and personal notebooks

For most of my professional software-development career, I have used notebooks to track my daily activities; to express goals; to write short prose essays journaling my thoughts about places, situations, and people; to keep track of “quotable quotes” worth remembering; and so on.

Yes, as a certified tech-geek, over the years I have also run around with digital organizers and other computers (tablets, phones) since my first PalmPilot. However, none of these has ever matched the ease of use or the durability of putting pen to paper and scribbling to-dos, goals, and thoughts. Yes, I have also ensured many times that these would end up in electronic supporting forms such as calendar events, emails, and OneNote entries when appropriate. I am not technology-agnostic. I merely seek to be as efficient as possible on getting something “down” and moving on.

I found this interesting article this morning (7 REASONS WHY SUCCESSFUL PEOPLE KEEP A NOTEBOOK) and found that it strongly resonated with my personal use of notebooks. As I have aged, as many also discover, my memory is not the same as it was when younger.

Notebooks help my overflowing brain by permitting my thoughts to be more digested and condensed through the process of thinking through what I am going to write. They help me hit daily, weekly, and lifetime goals through the process of review. They contain broad swaths of thoughts which may not be important enough to recollect day by day, but when the needed can be found and reappropriated. I also find that my notebook, in the way I set out the pages, help me to place fragments of thoughts adjacent to something so that I can return at the end of the day or the end of the week and dig deeper.

An example of this may be a discussion of a software issue with my manager: I note that we are having a meeting in the active day log and adjacent to it on the left-facing page, I may note words, small snippets of incomplete thoughts and conversation, and possible to-do items to process later. Of course, another use for my left-facing page may be to cartoon something funny or cynical about that day or to make some significant note about something of historical interest to me. For example, my father-in-law passed away a few weeks ago and so in my personal notebook, I pasted my hospital pass from that last visit and made a couple poignant notes. Here is another (less morbid) one from my work notebook at Christmas last year. I grabbed one of our House Mouse Design stamps and popped it on a page. Why I chose to not colorize it escapes my memory at the moment, but maybe because I was busy trying to finish off a ticket at work.

House Mouse reminder of the Christmas season.

As in most pursuits, there are lots of wars fought over brands and types of notebooks and pens. It is amazing how many battles are engaged concerning the only proper pen for a notebook is a fountain pen (as much as I like writing with one occasionally), or fine tipped Sharpies, or you name it. Other conflagrations circle around the brands of notebook. Some camps lean only on Moleskine, others on Leuchtturm, yet others on Miquelrius, and on and on ad infinitum. Personally, I have used many brands of notebooks and have been mostly satisfied by most. For the last six or so years, I have nested on Leuchttum 1917 A5s with dotted pages due to convenience and that they are (for me) the right size and priced right. I will write in them with my daily Tul pens, Sharpies, highlighters of various brands, pencil, my Rotring multi-function’s ballpoints and pencil, or whatever I can use to make the appropriate marks.

The key is to maintain a notebook (any notebook) and to develop a system (any system) that works for you. All other concerns should fade to the background. After all, it is your notebook and you use it to meet your goals, so why get yourself hung up with useless battles. There are lots of systems for using and keeping notebooks. My pattern is to save the initial pages for “Quotable Quotes.” Then I track my days in the bulk of the notebook, always using only the right-side pages for tracking tasks, meetings, goals, etc. while reserving the left-pages for random notes, scribbles, illustrations, and such. At the back of the notebook, I track more serious long-term goals and training notes. Every time I start on the next notebook, I begin by copying my “Quotable Quotes” into the new book (and I remix their order to keep them “fresh” between notebooks) and reserving a reasonable number of pages for their expansion. The same is true of my remaining long-term goals at the back of the notebook.

Sample of my Quotable Quotes with my chicken-scratch handwriting!

Another interesting side-effect of keeping a notebook is its historical durability. I wrote about the dangers facing future historians as digital material disappears and digital standards for documents and images become hopelessly obsolete (Future Challenges Facing Historians). Keeping concrete notebooks serve to maintain a record of your life and your history for future generations. Of course, the logical extension of this is to go beyond a mere notebook and keep a more comprehensive diary. Again, it is your notebook so if you prefer to keep it as a diary, then so be it! In my estimation, the key is to get the most out of the tool and have as much fun as you want to in the process of using it!

If you use a notebook, drop a comment and share any ideas that may be helpful!

Coniston Lake, an oil painting

I finally finished the one and only canvas that I started this year. Amid the work-at-home, high-pressure sprinting for Covance, and all the fun-and-games Covid brought, I decided to set up my easel and try to get my head back into something artistic for a while. I had been influenced by the complexities of a background on my “work” machine by a Mark Nelson which he had titled Coniston Water.

I did look around to see if I could find and contact Mr. Nelson but since I could not find him, and I was interpreting his work (along with countless other photos of the same area) for pleasure and not for profit, I went ahead and painted it. As you can see from this shot, it is a rather intriguing scene to paint and would definitely stretch me a bit.

Mark Nelson, “Coniston Water”

I started the oil painting at the end of August on a 24″ x 12″ canvas and unfortunately, due to the pressures of work life, it did not get completed until today. At least I can say that I got it done before the end of the year! Anyway, I wanted to try a few techniques. First, I wanted to do a prep using acrylic to establish a depth to the sky and the foreground water area. The sky has a warm brilliance that I wanted to come through and the foreground water is shallow and filled with dark pond weeds below the surface.

Once the base had dried, I started to work on the foreground and the distant elements. In order to bring forth the dark weeds in the foreground water I both lightly tonked the oil paint with a paper towel and applied a little sgraffito to expose the darker substrate. The sgraffito also served to introduce some movement into an otherwise overly static composition.

The rest of the time was spent in laying in the rest of the structures and in simplifying the original composition to keep it interesting as a non-photorealistic painting. I cannot stress enough the value of knowing the subject, even if you have never been there. I studied the satellite views of this area on Google Maps, found other perspective shots on Flickr, and even read up a bit about Coniston and its appeal as a scenic location. Perhaps the opportunity may present itself in which I may be able to gaze on its three-dimensional beauty one day in a post-Covid world. Perhaps.

Anyway, to draw a long painting story short, the work has been completed. My New Year’s promise to myself in 2022 is to not lose momentum and to find the time to paint some more. If you are a regular follower of my blog, you know that I strongly believe that “right brain” activities help feed and enhance “left brain” activities such as software engineering (see https://claforet.wordpress.com/2009/12/18/pragmatic-wetware-and-my-journey-into-painting/ for more details).

Now only one thing remains for this article to be complete, the final piece itself. So, without further ado and with all my best wishes to you in the coming year, here it is.

Coniston Lake – Oil on Canvas, 24″x12″

Engineers Must Learn From History

In my recent encounters with newly minted software developers (and by “recent,” I mean within the last decade or so), I have been shocked by the fact that most do not have a solid grounding in the work of the past. Even more disconcerting, many are disinterested in understanding the foundations upon which their career path has been built. The lack of care and understanding of history and its proper place in creating and maintaining a civilization has somewhat spilled into the realm of engineering disciplines.

This is a sad state of affairs. Any scientific discipline, and engineering is scientific in nature, grows on the understandings and misunderstandings of the past. In fact, this is such a well-understood fact that we even have a saying in our language for it, “Don’t reinvent the wheel.” In other words, if we are engineering a solution that involves moving across a surface, we don’t have to start off at ground zero to figure out basics of efficient motion. We can, instead, address how to repurpose and embellish/decorate the wheel to address our specific needs.

As engineers, depending upon our niche discipline, we have the benefit of hundreds and even thousands of years’ experience expressed in formulas, algorithms, practices, and understanding. Armed with this rich toolbelt, astute engineers can rapidly converge their energy on solving the root problem facing them rather than cast around to relearn the lessons of the past. (By the way, I am referring to the engineering of new solutions and not the helpful learning exercises that help develop our understanding by rediscovering what was done in the past.)

Recently, while browsing some of “Uncle Bob” Martin’s lectures on YouTube, I came across several in which he pointed out that some “new” technology of today existed in some form or another “back in the day.” (see, for example, 55 minutes or so into “The Future of Programming” https://www.youtube.com/watch?v=ecIWPzGEbFc&t=3345s) The point of this is for each of us to understand that we can take the pros and cons of some older technology, modernize it, and enhance it to create the next new thing.

If we don’t know what existed in the past decades, we have to tread the same ground others have already been through, making some of the same mistakes, and maybe even ending up with a poorer product in the long run. This is why we engineers need to cultivate a discipline of reading and listening to the voices of the “old timers” who have already been there and done that and have the T-shirt! When a young developer scoffs at learning the history of their discipline, they doom themselves to a life of difficulty, frustration, and maybe even obscurity.

This also lays a requirement on the shoulders of every seasoned software engineer. How would the newly minted developers learn from the past unless they hear about the rich engineering history we each embrace. It is incumbent upon each of us to become mentors to that next generation and to actively pass along what we know and understand. And, while we mentor them, we must not miss the opportunity to learn some new tricks from them also, but I digress. Mentor, teach, pair-program, discuss the big problems of the past and how they were overcome. We must help these brilliant, young, and agile minds grasp these lessons so they may become powerful exponents of our discipline instead of mere anemic workers in the software world.

Working from Home and Unfair Tax Rules?!

(Free clipart from clker.com)

Another Covid year is beginning to wind down with all of the trauma and changes that this disease has brought into the lives of the world’s population. It has definitely taken a toll in the deaths of loved ones and also a financial one. My ongoing prayer is that God have mercy on us and deliver us from all of it in 2022.

Last year, at least here in the US, most corporate employees were sent home at the end of the first quarter and were expected to work remotely (at least those who did not, unfortunately, get laid off). Some companies tried their best to ensure that their employees had access to computing equipment, and a very small number of them actually provided communications and proper office furniture. How ever it was handled, the workforce moved into home offices or into improvised home office spaces. This state of affairs, by and large, continues even now (October) for a large population of workers.

There obviously is a nasty critter hiding in our collective walls. Dare I mention its name? Ok, twist my arm! Its Increased Costs. Yep, most of us diligent workers have offloaded costs from our corporate employers and now fund those costs out of our pockets. While we are grateful to keep our jobs, this is a tangible expense for all of us. Companies may not be in a position to actually refund these costs without incurring a further bloodshed in employee reductions. Yet, we still have these expenditures: Increased power draw for equipment and lighting, increased heating and cooling costs because we cannot change our thermostats during work hours (when we are normally at the office), dedicating physical office space in our apartments and houses, increased internet and phone charges because we need additional bandwidth and service, etc.

If we actually hosted a home-based business and filed taxes as self-employed business owners, the Federal government would have us covered. The Home Office Deduction exists specifically to provide relief to those whose office expenses are not covered by a company. This has been, in fact, a staple offering for all self-employed businesses for decades. However, if you maintain a home office and are employed by another company, this eligibility is gone into thin air…Pffft! The basic reasoning for this (under normal conditions) is as an employee, you are choosing to work from home instead of from an office or your company is paying for this convenience out of their pockets one way or another.

Yet, what about the situation that we are currently undergoing? We are not choosing to work from home. Thanks to mandates and public health concerns, even those who cannot stand working at home were not, and in many cases, still are not permitted to assemble in offices. Increased Costs slither through the walls of our home offices and there is no recourse for them. Effectively, each of us has taken a pay cut!

What I can’t understand (and I am far from being alone in this musing) is why did the IRS not pass a temporary measure to permit home-bound employees from being able to deduct the business use of their homes as long as they are required to not return to the office? They missed it last year (gee, they had 9 months to read the state of affairs back then) and it appears that they are missing it again for fiscal 2021. They could have introduced a variation of Form 8829 and required a document from employers certifying that the employee in question has not been permitted back into the office for the duration of 2021 or until some date in the year. I am sure that most HR departments can spit those out without much fuss.

Does the inaction of the IRS in this matter make sense to anyone? Are my assertions unreasonable somehow?

The Pragmatic Programmer, Rebooted!

It is a true statement that one of the influential books in my personal software development journey is “The Pragmatic Programmer.” My copy has been well-read and substantially highlighted in the almost 20 years it has been in my possession and many of its principles have become innate in my development process. In fact, it is one of the books I most recommend to mentees and peers just because it presents a good commonsense scaffolding that any developer can use to build their careers.

Of course, one of the problems with any software engineering book is that it ages at the speed of technology advancement. Twenty years of this advancement has served to render some of the examples and pressing-issues-of-the-time obsolete. While this is less of a problem for some of us “old timers” who have lived through the turbulent software years since the 1990s, it does affect those readers who are new to the development world.

However, have no fear! Dave Thomas and Andy Hunt have revitalized the material and delivered their new and improved 20th edition. A second bite at the proverbial apple is a good thing and in this case it permitted the authors to add depth to some of the concepts they originally coined terms for (e.g. DRY) and follow up with logical improvements to those principles that have arisen through the years (e.g. ETC). Every software engineer should have a core library of great books and this one still needs to be in it.

I could not wait for my copy to arrive last week, an early Christmas present to myself. No disappointments here. It has been enjoyable re-reading the book with the combined material and discovering that this book should be as effective for the next decades as the original was. My only remaining hope is that I might have the opportunity to catch up with Andy and Dave again sometime so they can sign this copy for me as they did to my original copy so many years ago at a No-Fluff conference! Maybe after COVID?