Trend Analysis: methods and problems
Lyn Firminger March 2003 Trend analysis: Methods and Shortcomings
Abstract Trend Analysis is used in primarily in marketing, business planning and strategic
planning at both a corporate and national level. It has a number of sub-methodologies; historical trend analysis, content analysis, cyclical pattern analysis, and the use of expert opinions such as Delphi. It should not be used as a stand-alone method but can be useful when balanced with other methodologies. Trend analysis informed by other Foresight methodologies has the capacity to give analysts a broader and deeper understanding of possible futures. Introduction The futures field and its methodologies are changing. New methodologies are emerging, and older ones revitalised. This paper explores trend analysis (TA) as a methodology that has some underlying sub-methodologies, some of which might be useful in different types of analysis. This is not an exhaustive survey but serves to give an overview of different methods. A trend is evident when some phenomenon is seen to have a specified general direction or tendency. Some trends, for instance those using population data relatively stable, that is, not likely to be dramatically affected by other events or phenomena, they are built on other long-term trends such as birth rate, life expectancy and habitation patterns, which change slowly over time. Trend Analysis (TA) is also used to predict business, consumer, and political trends at local to global levels. These are less stable and more likely to be impacted on by other trends or events. For instance, a trend toward democracy in Indonesia might be indicated by a growing middle class, but this trend could be slowed or dramatically changed by a rise in religious fundamentalism, the political „colour‟ of a new president, prolonged drought, regional instability and so on. One of the keys to TA is to think broadly and deeply enough about the possible interruptions to a trend. It is also necessary to try to understand the author‟s cultural and political perspective and thus their possible blind spots, as TA is not value neutral. “Pop” Trend Analysis TA has become increasingly important to both small and large businesses. The marketing strategy of a business, particularly a small business, is based on finding out what the trends are in a particular industry, plotting a possible birth to death trend curve, marking with an „x‟ where the trend is at present (rising, at its peak or falling) and deciding how much to invest in getting in to that market. This is more true of small businesses because very rarely are small businesses trendsetters as they do not have the market power. Much of the development of TA has taken place in the US. At one end of the spectrum are analysts who operate at a very surface level or who rely on straight-line trends, without considering interruptions to trends. One of the gurus of surface level TA in America is Faith Popcorn. Her trend „bytes‟ are available free of charge on her website. Her products are hour-long seminars, consultancy and books. Members of the public can join her talent bank for free. Members supply demographic information, and may be contacted to comment on an
embryonic trend. This is the same logic that is used by marketing focus groups. She calls her methodology „brailing the culture‟, which is a scanning of popular behaviour. She monitors the top ten of „anything‟, movies, music, TV, books, plays, products and fashion. Popcorn‟s particular genius seems to lie in her ability to give a name to something that is happening among large groups of people so they can begin discussing and evaluating these trends for themselves, while canny business people develop the products or services to nurture the trend. For instance, „cashing out‟ is a trend that involves opting for a simpler lifestyle. I have seen two substantial articles in popular magazines and heard a number of passing references to this now named phenomena. Naming it, has given people an opportunity to examine some phenomenon that is affecting their lives. This type of TA operates at a surface level and, while it is not likely to give anyone any profound insights into life, it appears to have its uses. There are many organisations and „gurus‟ willing to forecast what will happen in the coming year or decade. These are often business consulting companies that also sell CDs, books and articles and sometimes free newsletters. One of these is Futurist.com. The Principal became a professional futurist in 1980 after an academic career. Unfortunately he does not mention if he did any Futures training. You can look at the trends for 2004 on their website. The value of this site is that the view is wider than just economic; it includes quality of life issues too. For instance the “biggest story” for the year is the declining birth rates, but at least the question is posed - why is this a problem? The guru of straight line TA is John Naisbitt who first published Megatrends in 1982. There have been numerous editions the most recent in 1996. The original TA tool Naisbitt used was to analyse the content of the news, in newspapers. The premise of Megatrends (1982) was that „the most reliable way to anticipate the future is by understanding the present.‟ He did this mainly through monitoring local events and behaviour „because what is going on locally is what is going on in America.‟ In the course of this monitoring, he and his colleagues have come to the conclusion that (a) trends start in local communities not in New York or Washington and (b) trends are bottom up and fads are top down. The genesis of this method of content analysis began in the work that he carried out under the leadership of Paul Lazarsfeld and Harold Lasswell during the Second World War. They would analyse the content of German regional newspapers to get an understanding about how Germany was really faring in the war. Information on supplies, production, transportation and the food situation was secret, but by carefully reading local newspapers they got a sense of how many factories were being closed or opened, what their production targets were, and so on. They also found that the local papers would publish local casualty lists, even though this information was highly secret at a national level. The military continue to use this type of content analysis. What Naisbitt did in Megatrends, was to apply it commercially. There is one further crucial theoretical consideration in his method: The news hole in the newspaper is a closed system. For economic
reasons, the amount of space devoted to news in a newspaper does not change significantly over time. So, when something new is introduced, something else or a combination of things must be omitted... In keeping track of the ones that are added and the ones that are given up, we are in a sense measuring the changing share of the market that competing societal concerns command. He gives as an example the increasing amount of space being given to environmental issues and that being matched precisely in column inches by a decrease in news about civil rights. The trends in this first book were: the information society, personal value systems growing when potentially dehumanising technological innovations are introduced, long-term thinking, the decentralisation of power, self-help, participatory democracy, networked world, the growth in importance of the southern states of America and multiple options replacing either/or scenarios. There are a number of problems with this approach. Firstly, the content analysis devised during the war was not intended as a tool to predict future behaviour; it was intended to look beneath the news to see what was happening today. Secondly, newspaper reporting is very selective tending to prefer the sensational to the more substantial. Thirdly, it assumes that a trend will go through its lifecycle uninterrupted by sudden shocks. This assumption has Naisbett predict that Japan will replace America as „the world‟s leading industrial power.‟ Naisbitt‟s first take on TA was very US centred, in subsequent works, looking at Asian trends he incorporated a view of „out there‟ but filtered through the American press. Many of his trends have petered out, as one might expect, but he continues to be successful by replacing them with new ones. Naisbett views progress as linear, (onward and ever upward), and relatively impervious to external shocks.
A new development - trends and interpersonal development Just before leaving the „pop‟ arena it is worth noting that TA is now being used to as a life guide. There are several examples of this. On October 29 2001 an organisation called LifeCourse Associates asked the question „what will the new normal be?‟ They argue that America may be entering „the fourth turning‟; that is there is a cyclical pattern to history evidenced in the past five centuries of America‟s history. Each „turning‟ takes approximately 100 years. First comes a „High‟, most recently evidenced in the baby boomer generation, followed by an „Awakening‟ a period of consciousness, such as happened after the American President Kennedy was assassinated, with the fourth period being a „Crisis‟ the last of which was the 1930s depression. This cyclical rather than linear, view of time is more typical of Eastern culture than Western, and does have relevance for TA, as will be shown below. Paul Ray uses TA to define a new political grouping, „cultural creatives‟, who are, according to Ray, „in-front, deep green, against big business and globalisation and
beyond left vs. right.‟ He uses a survey he carried out in 1995 as the basis of his 2002 trend analysis of this new and potentially powerful political grouping. He argues that the traditional trend analysts, the political pollsters and campaign advisors use, fails in picking up subtle changes in society. The methodology is not explained in this paper but I did learn from contacting him that his sample was 1036 people who after being contacted by telephone (along with an unspecified number of others) agreed to fill out a long questionnaire regarding their values. The analysis remains murky but it appears that the only results he saw fit to publish were those supporting the values he attributes to the cultural creatives. Despite the use of factor analysis and bell-shaped curves this trend sub-methodology ultimately fails to convince because the sampling and analysis are a lot less than transparent and American politics is far more complex than is made out here. This type of TA purports to look beneath the surface but relies on selectively picking through history, leaving behind what is not appealing and using it to support a thesis (the existence of „cultural creatives‟) that in itself is analytically on shaky ground. Trend analysis as a business tool There are many more seriously presented TA consultancies, some are industry specific, others such as the Australian company IBISWorld cover all industry sectors in some depth. In Australia the need for business intelligence utilising trend data increased in the 1980s. During the 1980s there was a large increase in micro businesses, particularly in the service sector. The influx of businesses meant that a business had to link into trends to survive. During the following 10 years the business environment became even more chaotic, harder to predict. One consequence was that a raft of consultancies emerged to assist businesses identify trends. One, important and highly respected such organisation in Australia is Phil Ruthven‟s IBIS World. It collects information on industries using hard data sources such as industry turnover, employment, number of establishments, etc as a grid to plot rises and falls in the fortunes of that industry. This is supplemented with qualitative data; newspaper stories on companies in the industry, annual reports of public companies, etc. This information is then sold. Non-subscribing visitors to the website can get a one-page overview for free and regular readers of Business Review Weekly will get the encapsulated form of the trends, as there is a tie-up between the two organisations. The information can also be sourced for free at Victorian Government Business Centres. McKinsey and Company are a large US based, international Consultancy that work with larger businesses to develop business strategies. They also publish an online newsletter with some free information, and a quarterly journal. They will follow a number of „hot‟ trends over a period of time across a wide range of industries. There are many hundreds of consultancies offering TA as one of a suite of tools in business planning. Some purport to take the uncertainty out of business planning, but it is difficult to see how this might be possible using the sort of linear trend projections they appear to advocate. Several serious trend analysts have published „how-to‟ books on TA and
these can assist unravel some of the mysteries. Trend Analysis as a strategic planning tool Gerald Celente has written his book on tracking trend with an aim to profit from them. It begins with endorsements, mainly from newspapers and gains credibility by being distributed by the World Future Society. It claims to „go beyond the rosy scenarios in Megatrends 2000. The Institute of which he is a part started tracking trends back in the 1980s. This book has been written in the belief that most people react to trends and therefore fail to benefit from them - whereas smart people „proact‟ in the face of trends. That is, they „anticipate the future and act accordingly hopefully ahead of the one to three year timeline business and government use.‟ The sub-methodology used by Celente is newspaper analysis, but at a more sophisticated level than Naisbitt. He goes into some detail about how to read newspapers and mark them for future reference when you are reading huge amounts of information. Trends come from different groups in societies as well as from social leaders. It is important to have a „search image‟ when you are searching, that is one or more trends you are looking for, but these must be broad enough to encompass trends you might not have thought about to begin with. He illustrates this with a story: if you go into the jungle just looking for monkeys, you will not find snakes. Implicit in this is that if your expertise is too narrow, you may well miss something important. He argues that to be a trend, an event must „follow logically ... from causes that span a number of fields or disciplines (and have) ... at least two events to form a sequence.‟ The fields the trend must appear in are social, political and economic. Trends will have a long life and go through the phases of birth, maturity and death; fads on the other hand are short-lived and without social, political and economic significance. Celente is aware that if you are going to use newspapers you need to be discriminating in your use of them, he does not, for instance, use content analysis because so much in the newspapers is unimportant. This is considerably different to Naisbitt with his unquestioning acceptance of all the news that is printed within the news hole in the press. Celente gives a detailed description of his method: You have to screen out the headline news and the junk news and focus on the real news, the current events that are forming future trends. Real news rarely appears on television and it rarely makes headlines. It is usually in the middle of the better newspapers or in specialized publications. There you can find warnings of a crisis months or years before it makes headlines. The second part of the book is taken up with looking at some major trends; these are media, politics, the family, education, health, the environment, the military and the world. These areas are treated, naturally enough from an American perspective. It is interesting to look back at this 1980s view of what the trend drivers were: he sees a trend to globalisation as a move away from ideology. The new century would leave behind the ideologies of communism, fascism, Nazism, and fundamentalism.
Standing in 2004, we can say this hasn‟t happened yet. Michael Mazarr weighed into the trends debate with a book looking at trends to 2005 but with a proviso that we have around ten years to influence these trends so that the more positive rather than negative aspects prevail. He also takes a broad world view. He is involved with an organisation CSIS, the Centre for Strategic International Studies. Financial support for the book came from The McCormick Tribune Foundation, The Korea Foundation and the BP-AMOCO foundation. Knowing this assists in understanding his biases. He takes his starting point from Peter Drucker: Every few hundred years in Western history there occurs a sharp transformation. Within a few short decades, society rearranges itself - its worldview; its basic values; its social and political structure; its arts; its key institutions. Fifty ears later, there is a new world. The current transformation will affect the West and most of the rest of the world. The transformation is from an industrial to a knowledge era and will be driven by science and technology but social, political and psychological changes will follow and complement these driving changes. He draws on a wide variety of sources from Daniel Bell‟s 1970‟s work on post-industrial society, Charles Handy‟s notion of paradox (which permeates the book) and the evolution from modern to post-modern society, John Peterson‟s wild cards, and Paul Ray‟s cultural creatives, among many others. Mazzar discusses the idea of moving from a Newtonian image of the predictable universe toward the sort of holism evident in quantum mechanics and complexity theory. His inclusion of quotes from Tao Te Ching suggests a willingness to look at cultures other than his own. The book does not contain an explanation of the methodology or prescriptions for trend analysis, but it is evident that it derives from very wide reading. In the substantive areas of the book it is clear he draws on United Nations data, media stories, and many varied writers. The book is problematic in that it has an overly complex layout and the Chapter headings often do not give a sense of where one is heading. Nine themes underlie the book. Issues dealt with in the book include world population, population and immigration, the authorship of the knowledge era, knowledge and the reorganisation of work, and globalism, tribalism and pluralism. In the conclusion he remains optimistic we will overcome problems, and that there will be a synthesis that grows out of whatever crises accompany some of these changes. One of the areas of synthesis is between Western and Eastern worldviews. Neither will predominate but there will be „mutual absorbsion‟ between the two. Mazarr is much more reflective than other analysts and his book could provide a useful tool to help a researcher think through the various issues.
Trend Analysis and multiple futures The different TA methodologies discussed so far have been predictive to some extent trying to pick the trend or the future of something. This need not always be the case, TA can be used to set out different possible scenarios. Barney Foran and Franzi Poldy, two CSIRO researchers undertook a study into the impact of the size of Australia‟s future population on the environment, the physical economy, the national infrastructure and our quality of life. They used trend data going back some 50 years and then extrapolated this data to 2050 using three different population scenarios; high, medium and low population growth. From this they distil a number of possible dilemmas Australia may face in future. An appendix gives comments from the reference group. This is an important work and if the Government make a choice as to the population scenario to opt for, will be a useful tool for those involved in demographic research. The US Government has considerable resources to fund futures research. One such piece of research was published as Global Trends 2015: A dialogue about the future with non-government experts was the second such report developed by the National Intelligence Council. It was approved for publication by the National Foreign Intelligence Board under the authority of the Director of Central Intelligence. It looks at the world „from the perspective of the national security policymaker.‟ It engages with outside experts from academia and private organisations rather than relying on traditional classified sources of information. The project began in autumn 1999 and was published in December 2000 and builds on an earlier work published in 1997. The work identifies global drivers and estimates their impact on the world over the next 15 years - demography and natural resources, technology, globalisation and governance, likely conflicts and prospects for international cooperation and the role of the United States.‟ This is a large broad-ranging document with a focus on American security, as one would expect from its authorship. The process allowed for a rich informational input. There were workshops attended by Government and non-government specialists from a wide range of fields. Each workshop had a different focus, one „demography, natural resources, science and technology, the global economy, governance, social/cultural identities and conflict and identified main trends and regional variations.‟ The second workshop was a scenario planning exercise built on the information from the first workshop. It highlighted „key uncertainties, discontinuities and unlikely or “wild card” events and (identified) important policy and intelligence challenges.‟ Thirteen conferences were held; each co-sponsored by the NIC and other government and private centres. Each had a different theme, for instance evolution of the nation state, trends in democratisation, American economic power, and alternative futures in war and conflict, to name a few. A draft report was then put together and circulated to outside experts and the final report developed. The paper begins with drivers and trends, looks at key uncertainties and then key challenges to governance. The next „discussion‟ section covers the substantive areas of population, environment, etc. This is followed by a
thumbnail sketch of the major regions. The four alternative global futures are contained in the Appendix along with a matrix of the drivers (population, resources, etc) and how they might develop under each different scenario. While this work has a focus on the US, its underpinning information can be used to analyse trends in other countries too. The changing methodological paradigm There is a growing need for TA and other Foresight methodologies that are capable of allowing for multiple possible futures. In some cases these can then act as organisational discussion starters about preferred futures or as input into Scenario Planning exercises. Drivers of this change include growing global uncertainty the shortening of the business cycle and a shift from an industrially based society to one based on knowledge and information. Underlying the shift in Futures thinking is the decline of Positivism as a philosophy underpinning social discourse and the rise of modernism and post modernism. A recent survey of the life work of Wendell Bell shed some light on this changing futures paradigm. For Bell an assertion of a possible future - or anything else - must be accompanied by evidence, the assertion must be able to stand up to the rigour of attempted falsification of the hypothesis. Bell‟s intellectual roots are in the American „scientific‟ sociological movement. Bell has acknowledged that critiques of positivism have helpfully corrected an uncritical acceptance of positivist science‟ but if taken to a logical conclusion „“not only would causality, determinism, necessity, objectivity and rationality be abolished, but also humanism, liberal democracy, responsibility, truth itself and, we can add, futures thinking.”‟ TA has embedded into it the sort of conjectural knowledge that „incorporates justified beliefs about the future‟ typical of this tradition according to Stevenson. Is TA doomed to remain within the positivist or critical realist paradigm as Bell calls his move beyond positivism? Does it have any relevance in Future Studies today? TA certainly suffers from all the shortcomings Richard Slaughter found in his critique of the Futures domain as it stood in the early 1980s. It tends to be superficial, „it fails to recognise the roles of language, power and embedded social interests, it lacks understanding of its own sources and grounding‟, it is appropriated by the powerful, it suffers „overconfidence in easy prescriptions‟ and it is not open „to other ways of knowing.‟ Slaughter is concerned with the emancipatory role of futures, its role in reconnecting the inner world (not taken into account in the positivist tradition) with the outer world. Rescuing Trend Analysis for the Future Trend Impact Analysis (TIA) is a methodology that explicitly deals with unexpected events as part of a quantitative based TA. The method allows for the systematic treatment of possible future events, be they technological, political, social, economic or value-oriented. Expert judgments are sought about the probability or an event as a function of time
and its expected impact on the trend under consideration. To begin with, historical data is selected and a surprise-free extrapolation is developed using a curve-fitting algorithm, as used by Foran & Poldy. At this point the judgment and imagination of the researcher is critical. A list of possible potential events is prepared. „These events should be plausible, potentially powerful in impact and verifiable in retrospect.‟ The source of such a list might be a Delphi study, some form of other informal consensus among experts or a literature search. The probability of the event is estimated then an estimate is made of when (or the duration of) the start of the effect, followed by the impact of the trend at its largest or when the impact might reach a final or steady-state level. Alternatively one might try to estimate the magnitude of the event the largest impact or the steady-state impact. The paper discusses TIA software that gives a quantitative analysis of probability of occurrence and the impact on the trend. This may be useful to Managers wishing to estimate the probability that strategic targets might be met over a number of years. A simple to follow example of the continued use of chlorine in bleaching paper is used as the example. Gordon assesses the strengths of TIA as that not only does it allow you to say “Here‟s where I think the variable will go in the future,” it also allows you to add, “and here are the events I‟ve taken into account.” He likens this to constructing a scenario, just as the NIC did in their study. A further advantage of TIA to the strategic planner is that once the possible events are identified they can be built into a scanning framework for the trend.
A number of other Futures techniques might also be used to enrich TA. Wild cards is one, there is an increasing recognition that the wild cards developed by John Peterson need further development to be of continuing use to those outside America. Mark Justman took Jim Dator's four images of the future continuation, crash, slowdown, and transformation; and mapped them against Peterson‟s wild cards to enrich their possible usage. As we have seen above Delphi and Scenario Planning have also been used to enrich TA. Conclusion Trend Analysis is particularly useful in marketing, business planning and strategic planning. To carry out TA a researcher will probably have carried out an environmental scan and detected that there appear to be trends in some item of interest. Something is continuing to hold an ascending or descending pattern or when A happens B almost always follows. The sources of information used to follow up on the trend will vary with the trend under consideration. Sometimes news appearing in the tabloids may be a good way to follow a trend, usually other methods will be needed. Some of the analysts surveyed here draw on surveys, historical analysis, conversations with experts, and so on. The difference between John Naisbitt and Michael Mazarr for instance lies in the diversity of inputs into the analysis. The critical question to ask at every juncture is „could anything happen now or at some point in the future to disrupt this trend?‟ Follow this with questions about the likelihood of an event occurring and its possible impact. Ask these questions through various
experts; this could take the form of a literature search, an expert panel or a Delphi panel. Explicitly factor the results into the TA; the outcome will be that the researcher will have a broader understanding of what is happening now and the possible impact on future events.