Charting 101

My day job as a trader is to evaluate charts as the visual representation in the change in prices, or as my investing partner call’s it, “tea leaf” reading.

Charting is just measurement of past prices, not a prediction of future results (I leave that to the economists) and I’ve seen many charts used by product providers, financial planners and spruikers alike that portend to say “x” by changing the parameters of the chart, which is really saying “?”

The classic one is the All Ordinaries since 1901 highlighted with all the “events” that the stock market brushes aside on its inexorable rise. Although this one by Fidelity reflects the etymology of their brand:

30 years of the All Ords - from Fidelity International

Here is an even better chart, constructed by Data Diary, that highlights the significant sideways periods experienced by markets.

Now, there is a particular chart that has been used repeatedly by Rismark and highlighted in a recent article as justification of “popping a housing bubble fantasy”, by inferring that the dwelling price to income ratio is stable and a reflection of a “secular decline in interest rates”.

Here is the chart in question (I’ve used an earlier version because this one includes an overlaid second series of data)

There are 3 major problems here:

  • The scale of the chart is linear, not semi-log, which means it is incorrectly displaying an exponential change in values (going from 2x to 3x is a 50% change in the ratio, whilst 3x to 4x is only a 33% change)
  • Because the starting value is approx 3 times, the terminal Y-value of zero times is a nonsense (it should be 1 at a minimum to reflect the reality that you need some income to purchase a house)
  • Combined, the use of two horizontal lines to imply a “stable” ratio implies the dataset is stable without growth, with only a structural change in the middle.

Linear and Semi-Log
A linear chart is usually used when the nominal change in underlying values does not have an exponential relationship, that is, both series are of a similar scale/size. A semi-log chart should be plotted when a large series of values (e.g a changing ratio) is plotted against a restricted series of values (e.g time).

In the chart above, the time series on the X-axis is restricted – its simply a quarterly change in time repeated. The Y-axis series however, oscillates between approx. 2.7 times and 4.5 times – or a near 100% change in an exponential ratio. This is obfuscated by using a terminal Y-value of zero, when common sense would dictate that you can’t buy a house with zero times your earnings. Further, the ratios below approx. 2x and above 5X have not occurred for the entire time series so why is the lower and upper boundary so wide as to compress the chart?

Therefore a nearly 100% change in the price-to-income ratio is displayed within only 25% of the vertical space.

A shiny example
Since the data above is restricted, I’ve found a similar time series and change in price in another market value to compare how data can be displayed.

Silver in USD per ounce from February 2004 to September 2007 nearly doubled – similarly to the price-to-income ratio in Rismark’s chart above.

Silver in USD per ounce - linear scale - click to enlarge

Plotted using similar boundaries and on a linear scale implies nice stable prices, a small structural change in the middle and an observation that silver is not expensive at all, at $14 an ounce.

But what about in semi-log scale?

Silver in USD per ounce Jan 04 to Sep 07


A clear, exponential increase in value as silver moves from $6 an ounce to almost $14 an ounce. If silver was a currency or ratio used by a central authority to measure underlying conditions, what chart best provides an indication of stability with a structural change or a tearaway trend?

Summary
Using charts to provide observers with a visual impact of the change in underlying data is a double edged sword. On the one hand, it can provide a quick summation of an obvious trend, or a lack of trend thereof. On the other hand, it can be used to obfuscate the truth by confusing the observer, even though the data (a near doubling in income required to purchase a dwelling) is clear.

Latest posts by Chris Becker (see all)

Comments

  1. Very enlightening post Prince. I must admit that at first glance I probably would not have noticed the deceit in that Riskmark chart. There are so many different ways to lie with statistics…

  2. Charles Ponzi

    I don’t need graphs or statistics to know that either my wage must double or house prices crash before it makes sense for me to purchase a house in Australia.

  3. Just to be clear, I am only implying obfuscation, that is “to render unclear or unintelligible”.

    There is no doubt that in recent years the required income to service the purchase of a dwelling has nearly doubled, according to the Rismark methodology.

    The RBA released their chart pack today, as I am extremely busy, I will post on it tomorrow. They too are guilty of using linear scales to reflect a very large change in value in the house price series, even though they use a semi-log scale for exports/imports change over time….

  4. The basic rule of this game is to choose the scale to maximise the visual impact for the sake of your argument. Spent a long time in the research industry, used to teach my PhD students that as lesson #1. Be extremely wary if you see a log scale on both axes…

  5. The 2 silver charts are over different time periods. That is where some of the obfuscation comes from.

    I understand that a semi-log scale is best used to show an exponentially growing data series, however it is not clear why it’s best for ratios between two exponentially growing data.

    • You’re right about the semi log chart, I zoomed out too much when I was moving the boundaries.

      Updated with the right chart. Thanks

      There are not two separate exponentially growing data series – if there was, the chart would be log-log, not semi log.

      Time series are not exponential, they are restricted.

      • Sorry I wasn’t clear. The two exponential series are (1) dwelling prices and (2) incomes. The ratio between those two is not exponential, I would have thought..

      • You’re right again David, and MB have gone over this before.

        There is a direct correlation, because the Rismark data includes the change in house prices in its disposable income calculation (from the National Accounts).

        Also, considering that (until recently) the Sydney market was very flat and comprises a significant portion of the households in the country, the “flatness” of the later half of the data is due to the other capitals catching up.

        I would like to see a complete series showing the individual capital cities price to income ratio.

        • Hi Prince. I love this analysis. Thanks for bringing this up.

          I have to disagree, however, with both David and you in your post analysis.

          Here’s why: I went over the entire RP Data report last night. The thing that surprised me the most is that most charts were about the rate of change on an annual basis. In fact, I can’t remember now if I saw a proper price chart.

          This point is important because when you analyse the slope of a price chart, you engage in studying the rate of change, that is the spear at which prices change (the first derivative).

          When you plot the rate of change of prices and you analyse the slope you engage in the study of how explosive the trend might be. That is you look at the acceleration of prices (second derivative).

          Those indicators are known as momentum indicators and should never be the starting point when embarking in chart analysis.
          In summary, it doesn’t matter whether the curve is linear or exponential.

          I remember watching a lecture on why the exponential function was the most misunderstood and what consequences that had, several times. Someone posted a link to it here not long ago.

          I think it should be mandatory training for anyone publishing these reports.

          -gt

  6. Caveat emptor.
    Every sunken ship contains a room full of charts. Technical Analysis Rule.

    Sounds like Davey Jones does well setting traps painting the tape.

  7. I always thought that graph showed that price grew massively from 2000-2003. They used to be approx 2.8x income, and now they are, say, 4.1x. That’s a 46% increase of prices relative to incomes in 3 years.

    He suggests this is due to a structural change in interest rates, yet mortgage rates were about 7% in 1997 and still 7% in 2003 (with a bounce in the middle).

    Also, most of CJs analysis to justify current prices will start in 2003 to avoid this controversial period.

    • >Also, most of CJs analysis to justify current prices will start in 2003 to avoid this controversial period.

      Well since he’s completely eviscerated the use of any long term historical measure by The Economist (and other non-mainstream economists who realise the historical impact and build up of credit flows and housing over-supply) why go back more than 8 years?

      Its like stockbrokers that like to show how “over the long term” i.e from 1981 onwards, the stock market was awesome. They forget to show the 15 years before hand when it went nowhere (1% p.a nominal).

  8. There is also a difference between the charts in the way the trend lines have been fitted to the data. In the second chart we have a line of best fit through all data points over the whole period of observation. The line of best fit then suggests continued growth. In the first chart you could also have a line of best fit and it would again show growth over the whole period. The difference in conclusion is not so great with this fairer treatment of data. However, you decide to treat data either side of the main growth point separately and to emphasise no growth set the lines horizontal through the peak in each period.
    Perhaps the lesson or point being made is that data can be manipulated to show a conclusion already reached. It follows Mark Twain’s observation about lies damned lies and stats

  9. There is no reason to choose a semi-log scale for a ratio unless it is a naturally exponentially increasing ratio – which a price/income ratio is not.
    There is also no logic in saying “it should be 1 at a minimum”. The ratio can be a half or whatever. The logical lower limit is zero.
    There’s an example of a highly deceptive time series chart here:
    http://macrobusiness.com.au/wp-content/uploads/2011/07/hpihf.jpg
    which really should have a vertical log scale and well illustrates the point you are making.
    Notice that a single vertical division represents a 100% change in the variables early in the chart but only a 10% change later in the chart – thus masking the enormous lack of adjacency between the two plotted variables in the 1990s and thus giving a false impression of the relation between the variables.

    • I tend to agree that this is one time that a log scale is not beneficial, and that the scale should in fact start at zero (to allof for ratios that are a fraction of one). The only criticism I have is that the scale is a quite generous, masking the severity of the change. Eg, if the y axis finished at 5x instead of 8x the viewer would get a better impression of the scale of the change that occurred from 2000-2003.

        • I also would suggest that there is context around the graph. If I recall, it was used to demonstrate how far off Australia’s price/income ratio is to other countries whose ratios are in the 5-8 range. Hence the reason for the compression of the axis.

          If the point of the graph was to examine the scale of change from pre2000 to post2003, then the axis scale seems inappropriate.

    • Suzi, In the interest of full disclosure, are you one of the four and a half PhDs at Rismark “International”?
      .
      PS: Go on, ask me why I said “four and a half PhDs” 🙂

      • What a rude post. I understood from UE that attacking the man/woman was a banning offence here?

        • Don’t worry, I have endured a few arrows since being here – my recommendation, persist and rise above. Eventually, all is OK.

          Best of luck. Your arguments are too complex for me at this hour of night (or any, my many detractors would say).

          Cheers.

    • Suzi if I pay 2 times my income for a house five years ago, but 4 times my income now, that’s not a naturally exponential ratio?

      I’ve now paid double – not incrementally.

      Also you can’t pay zero times your income to buy a house and not even 1 times, so why use as starting point?

      Even if it wasn’t plotted on semi-log scale, the chart obfuscates by compressing the data into a tighter range, to hide the fact there has been a huge change over time.

      As for the other chart, yes it probably could be plotted semi-log, but in this case its comparing two values which are highly correlated to each other.

      If you plotted it semi log they would be almost a 45 degree line – i.e showing an exponential rise in house prices alongside an “inexorable rise” in housing finance – which has now peaked.

      • No, it’s not. I think you’re perhaps using the common usage of the phrase to apply to statistics, where it has a clearly defined usage. An exponential equation would not fit the chart very well, as the residuals would be non-random (i.e. greatly positive at the start and greatly negative at the end).

        A linear equation (rather than exponential) will always double over a long-enough period of time when the slope is positive. The period of time just depends on the slope of the line.

  10. Tulip Flipper

    It’s really is interesting how you can use a chart to represent an idea, pandering to an audience who may be thinking a certain way (ie. greed/self serving), essentially as a sales tool.

    Nice post, Prince.

    • Hey, TF, did you read ‘Tulip Fever’ by Deborrh Moggach – brilliant and well worth the two or three bucks on amazon used.