Category: Food for thought

  • Thermocline of Truth

    This article was written in 2008. We are in 2021 now. And this article is still relevant.

    A thermocline is a distinct temperature barrier between a surface layer of warmer water and the colder, deeper water underneath. It can exist in both lakes and oceans. A thermocline can prevent dissolved oxygen from getting to the lower layer and vital nutrients from getting to the upper layer.

    In many large or even medium-sized IT projects, there exists a thermocline of truth, a line drawn across the organizational chart that represents a barrier to accurate information regarding the project’s progress. Those below this level tend to know how well the project is actually going; those above it tend to have a more optimistic (if unrealistic) view.

    Several major (and mutually reinforcing) factors tend to create this thermocline. First, the IT software development profession largely lacks — or fails to put into place — automated, objective and repeatable metrics that can measure progress and predict project completion with any reasonable degree of accuracy. Instead, we tend to rely on seat-of-the-pants (or, less politely, out-of-one’s-butt) estimations by IT engineers or managers that a given subsystem or application is “80% done”. This, in turn, leads to the old saw that the first 90% of a software project takes 90% of the time, and the last 10% of a software projects takes the other 90% of the time. I’ll discuss the metrics issue at greater length in another chapter; suffice it to say that the actual state of completion of a major system is often truly unknown until an effort is made to put it into a production environment.

    Second, IT engineers by nature tend to be optimists, as reflected in the common acronym SMOP: “simple matter of programming.” Even when an IT engineer doesn’t have a given subsystem completed, he tends to carry with him the notion that he whip everything into shape with a few extra late nights and weekends of effort, even though he may actually face weeks (or more) of work. (NOTE: my use of male pronouns is deliberate; it is almost always male IT engineers who have this unreasonable optimism. Female IT engineers in my experience are generally far more conservative and realistic, almost to a fault, which is why I prefer them. I just wish they weren’t so hard to find.)

    Third, managers (including IT managers) like to look good and usually don’t like to give bad news, because their continued promotion depends upon things going well under their management. So even when they have problems to report, they tend to understate the problem, figuring they can somehow shuffle the work among their direct reports so as to get things back on track.

    Fourth, upper management tends to reward good news and punish bad news, regardless of the actual truth content. Honesty in reporting problems or lack of progress is seldom rewarded; usually it is discouraged, subtly or at times quite bluntly. Often, said managers believe that true executive behavior comprises brow-beating and threatening lower managers in order to “motivate” them to solve whatever problems they might have.

    As the project delivery deadline draws near, the thermocline of truth starts moving up the levels of management because it is becoming harder and harder to deny or hide just where the project stands. Even with that, the thermocline may not reach the top level of management until weeks or even just days before the project is scheduled to ship or go into production. This leads to the classic pattern of having a major schedule slip — or even outright project failure — happen just before the ship/production date.

    The Wetware Crisis: the Thermocline of Truth

  • Momentum Investing

    Deepak Shenoy succinctly describing what momentum investing is.

    In the financial world, our job is to react to change. Not to predict it, because predictions are folly. You can predict a hundred things, and one of them is bound to happen, so you can say you were right. No one sees the ashes of the ones that went wrong.

    Reacting is easier. You wait till something becomes a trend, enough for it to sustain. And then you get in. You’ll never get in so early you can be called a soothsayer. You’ll get in after enough people have done so, at a higher price. And in the same way, you’ll be able to get out much after the top, but before the really bad damage hits. In the markets, we say – you might give up the first 10% or the last 10%, but at least you can get 80% of the trend.

    In so many ways, that is what momentum is. The trend is established and accelerating. A stock makes a new all time high. And if you just systematically buy those stocks, you’ll win on a few of them that go on to make the 2x, 3x, or indeed, 4x returns that some stocks have seen. And when you lose, you lose 10% or so, and one big winner makes up for enough losers.

    Wealth Letter July 2021: Following the Disruption

  • You need to cut yourself some slack

    A wonderful article by Shane Parrish on why slack is so important. It’s so important there is a book about it–Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency.

    …imagine one of those puzzle games consisting of eight numbered tiles in a box, with one empty space so you can slide them around one at a time. The objective is to shuffle the tiles into numerical order. That empty space is the equivalent of slack. If you remove it, the game is technically more efficient, but “something else is lost. Without the open space, there is no further possibility of moving tiles at all. The layout is optimal as it is, but if time proves otherwise, there is no way to change it.”

    Having a little bit of wiggle room allows us to respond to changing circumstances, to experiment, and to do things that might not work.

    Slack consists of excess resources. It might be time, money, people on a job, or even expectations. Slack is vital because it prevents us from getting locked into our current state, unable to respond or adapt because we just don’t have the capacity.

    Efficiency is the Enemy

    And at the end.

    Amos Tversky said the secret to doing good research is to always be a little underemployed; you waste years by not being able to waste hours. Those wasted hours are necessary to figure out if you’re headed in the right direction.

    Efficiency is the Enemy

    Wow!

  • What’s in a name?

    It’s often said that the stock market’s main oxygen comes from sentiments and the share price has hit the roof for a company that has the word ‘Oxygen’ in its name despite its business having nothing to do with the life-saving gas — something in high demand due to the Covid-19 pandemic.

    The share price of Bombay Oxygen Investments Ltd hit its upper circuit limit ₹ 24,574.85 apiece at the BSE on Monday, with the maximum permissible gain of 5 per cent due to the stock being under surveillance. The same is the case with some other little-known stocks with ‘gas’ or ‘oxygen’ in their new or old names and all of them are being probed for any possible foul play.

    What’s in a name? A lot for stock market, if it’s ‘Oxygen’ in Covid-era!

    I mean… I can’t even… Kaun hain ye log? Kahan se aate gain ye log?

  • Embrace the Grind

    An article worthy of putting on a refrigerator magnet so that I can read it every day.

    I often have people newer to the tech industry ask me for secrets to success. There aren’t many, really, but this secret — being willing to do something so terrifically tedious that it appears to be magic — works in tech too.

    We’re an industry obsessed with automation, with streamlining, with efficiency. One of the foundational texts of our engineering culture, Larry Wall’s virtues of the programmer, includes laziness:

    Laziness: The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful and document what you wrote so you don’t have to answer so many questions about it.

    I don’t disagree: being able to offload repetitive tasks to a program is one of the best things about knowing how to code. However, sometimes problems can’t be solved by automation. If you’re willing to embrace the grind you’ll look like a magician.

    Embrace the Grind

    Read on for some amazing examples.

  • Dependency confusion

    A simple loophole discovered by Alex Birsan and Justin Gardner

    Last year, security researcher Alex Birsan came across an idea when working with another researcher Justin Gardner.

    Gardner had shared with Birsan a manifest file, package.json, from an npm package used internally by PayPal.

    Birsan noticed some of the manifest file packages were not present on the public npm repository but were instead PayPal’s privately created npm packages, used and stored internally by the company.

    On seeing this, the researcher wondered, should a package by the same name exist in the public npm repository, in addition to a private NodeJS repository, which one would get priority?

    To test this hypothesis, Birsan began hunting for names of private internal packages that he could find in manifest files on GitHub repositories or in CDNs of prominent companies but did not exist in a public open-source repository.

    The researcher then started creating counterfeit projects using the same names on open-source repositories such as npm, PyPI, and RubyGems.

    Every package published by Birsan was done so under his real account and clearly had a disclaimer in place, stating “This package is meant for security research purposes and does not contain any useful code.”

    Researcher hacks over 35 tech firms in novel supply chain attack

    Now here’s the kicker

    Birsan soon realized, should a dependency package used by an application exist in both a public open-source repository and your private build, the public package would get priority and be pulled instead — without needing any action from the developer.

    Researcher hacks over 35 tech firms in novel supply chain attack

    Yikes!

  • The Kierkegaard Conundrum

    The english version of “Shaadi ka laddoo jo na khaaye woh pachtaye, jo khaaye woh bhi pachtaaye

    If you marry, you will regret it; if you do not marry, you will also regret it; if you marry or if you do not marry, you will regret both; whether you marry or you do not marry, you will regret both. Laugh at the world’s follies, you will regret it; weep over them, you will also regret it; if you laugh at the world’s follies or if you weep over them, you will regret both; whether you laugh at the world’s follies or you weep over them, you will regret both. Believe a girl, you will regret it; if you do not believe her, you will also regret it; if you believe a girl or you do not believe her, you will regret both; whether you believe a girl or you do not believe her, you will regret both. If you hang yourself, you will regret it; if you do not hang yourself, you will regret it; if you hang yourself or you do not hang yourself, you will regret both; whether you hang yourself or you do not hang yourself, you will regret both. This, gentlemen, is the sum of all practical wisdom.

    Either/Or: Kierkegaard on Transcending the Tyranny of Binary Choice and Double Regret
  • Anonymous corruption

    I recently watched the episode ‘Diamonds Aren’t Forever’ of Bad Billionaires: India on Netflix. This statement from James Crabtree stood out for me.

    There’s a very clear double standard in the way that we view corruption in the world. People look at countries like India and say, “these countries are corrupt”, but, actually, much of the worst corruption would not be possible without seemingly respectable international finance. Somebody like Nirav Modi needed international banks in order to funnel his money abroad. He needed international lawyers. He needed tax havens in order to host his shell companies. And so, the corruption, the grand corruption you see in countries like India works hand in glove with a different and much more anonymous form of corruption, which is entirely created in rich countries like Britain or Switzerland and America.

  • Children give the first four years of your life back to you

    It’s also a history lesson. The first four years of your life. Do you remember them? What’s your earliest memory? It is fascinating watching your child claw their way up the developmental ladder from baby to toddler to child. All this stuff we take for granted, but your baby will painstakingly work their way through trial and error: eating, moving, walking, talking. Arms and legs, how the hell do they work? Turns out, we human beings are kind of amazing animals. There’s no better way to understand just how amazing humans are than the front row seat a child gives you to observe it all unfold from scratch each and every day, from literal square zero. Children give the first four years of your life back to you.

    On Parenthood

    A wonderful article for all the new parents and the ones who are on the fence about becoming a parent.

  • Myside bias

    As we sit here over six months after the initial lockdown provoked by COVID-19, the United States has moved out of a brief period of national unity into distressingly predictable and bitter partisan division. The return to this state of affairs has been fuelled by a cognitive trait that divides us and that our culture serves to magnify. Certainly many commentators have ascribed some part of the divide to what they term our “post-truth” society, but this is not an apt description of the particular defect that has played a central role in our divided society. The cause of our division is not that people deny the existence of truth. It is that people are selective in displaying their post-truth tendencies.

    What our society is really suffering from is myside bias: People evaluate evidence, generate evidence, and test hypotheses in a manner biased toward their own prior beliefs, opinions, and attitudes. That we are facing a myside bias problem and not a calamitous societal abandonment of the concept of truth is perhaps good news in one sense, because the phenomenon of myside bias has been extensively studied in cognitive science. The bad news, however, is that what we know is not necessarily encouraging.

    The Bias that Divides Us

    A fascinating article by Keith E. Stanovich on what myside bias is. Highly relevant in these divided times and may help in understanding other person’s perspective. A bit more succinct explanation later in the article.

    …people evaluate the same virtuous act more favourably if committed by a member of their own group and evaluate a negative act less unfavourably if committed by a member of their own group; they evaluate an identical experiment more favourably if the results support their prior beliefs than if the results contradict their prior beliefs; and when searching for information, people select information sources that are likely to support their own position. Even the interpretation of a purely numerical display of outcome data is tipped in the direction of the subject’s prior belief. Likewise, judgments of logical validity are skewed by people’s prior beliefs.

    The Bias that Divides Us