From Model Building to Research Mapping

by Dr. Alex Liu

 
  • Preface 

  •  
  • I. Alex's Journey of Method Learning

  •  
  • II. Limits of Model Building Approach 

  •  
  • III. Introduction to ResearchMap

  •  
  • IV. Success Cases of Using ResearchMap

  •  
  • V. A Note of Causality Automation

  •  
  • VI. A Quantitative Research Example:
    Political Participation and
    Dissatisfaction with Democracy

  •  
  • VII. A Qualitative Research Example:
    ICT and Democracy in China

  •  
  • VIII. A Measurement Example:
    Index Correlation and Reliability
    of Democracy Measures

  •  
  • Conclusion: From Model Building
    to Research Mapping

  •  
  • References

  •  

 


I. Alex’s Journey of Methodology Study

 

When I studied engineering in the Northwestern Polytechnic University from 1978 to 1982, I was attracted to social studies due to my strong wishes to participate in China’s social reforms. Because of my strong social participation intention and engineering background, I hoped all good social researches to be scientific with results actionable. At that moment, to me a college student who loves mathematics, it seems to model social systems mathematically provides a lot more challenges and fun than to model engineering systems. Politically, I still trusted China’s political systems and respected its leaders. However, I was puzzled by the many horrible decisions China’s great leaders had made in the past. Through a lot of reading, I came to conclude that rigorous and scientific social studies were what China needed the most at then. I believed that we should be able to construct scientific social theories similar to physics and to run social management as running engineering projects. Therefore, in 1981, I decided to give up my engineering career and started my journey of exploring methods of studying social systems.

As guided by the thinking mentioned above, in 1983, I switched to sociology and entered Beijing University as a graduate student of sociology. At then, the Beijing University sociology department favored empirical research methods that my classmates and I got a lot of assignments to learn and practice questionnaire survey and data analysis. We even got to conduct surveys for the State Council’s Economic Reform Committee, and to propose public policy ideas. However, results produced from these surveys were not satisfactory, as they never led to any good solutions but only to some useful hints. I was a little disappointed by these results, and thought they were caused by my lack of advanced empirical research skills so that I needed to learn more.

Therefore, in 1986, I entered a Ph.D. program at Stanford University where I took many statistics and computing courses besides sociological work. With the data I collected from China, I tried some of the advanced data analysis tools I learned. But, it came out to me these advanced tools were still not enough to solve my problems that I started casting doubts on the prevailing empirical research practice. One day in Stanford Bookstore, I came cross a book Making It Count by professor Stanley Lieberson and immediately got attracted to it. In its introduction, professor Lieberson wrote:

“There were many failures before human successfully learned to fly. After watching birds flap their wings, bold and adventurous individuals built huge wing like structures, leaped off cliffs, flapped their wings vigorously, and broke their necks. There are principles of flight to be learned from watching the birds all right, but the wrong analog had been drawn.”

Then, he went on to argue that “in similar fashion, our empirical approach to social behavior is based on an analog”, like building wing like structures to fly. I did not agree with him on many of his arguments made in the book. But the above analog he made became something I never forget.

With some doubts on the empirical social research practice, I still moved on to learn more of empirical social research, but my focus was on statistical techniques and computing tools for data analysis since I thought they will always be useful anyway. I took many advanced statistics courses and earned a M.S. of statistical computing from Stanford. In practice, I participated in the 1989 Beijing pro-democracy movement and got elected to be the founding president of the Independent Federation of Chinese Students and Scholars in USA after returned from China. Partially due to this special experience, I selected democratization as my main research topic and conducted a few empirical researches on it. The result of this work led me to produce a book Patterns and Results of the Third Democratization Wave. However, my research and my practice are separated from each other. My research did not produce any meaningful guidance to my practice that cast a lot more serious doubt on my believing in empirical social research than ever. I felt this might be resulted from my restrained to sociology and to quantitative analysis. In other word, interdisciplinary approach and quantitative mixed with qualitative methods may cure my problem. Here, I mean my way of collecting data was restrained to the quantitative way and sociological way. In general, I did not agree with the way most scholars viewed the qualitative data. In terms of data, I think that qualitative and quantitative are the same, and both can be processed by computers.

Started in 1993, I jumped into health economics first, then to the studies of social networks and globalization, then to the fields of entrepreneurship and international business, in order to be interdisciplinary. All these studies, especially the study of entrepreneurial process, are fascinating to me. However, more work led to more doubts for me, especially when I had difficulty in dealing with the subjectivity issue. The entrepreneurial process is especially stimulating to me. At the beginning, entrepreneurs often do some market research formally or informally. Then, they act on what came out from this research. But the results out from this process are often different from what suggested by the early research. However, this does not prove the research is wrong, but only to show the power of entrepreneurs’ will and participation. This led me to some thoughts similar to George Soros’s reflexivity. Also, the more work I do and the more I got to see the huge difference between natural science and social science, as I look for actionable knowledge out from research. Definitely, additional human thinking is always needed together with what research suggested to enable a good social action.

From 2000 to 2005, I got a few opportunities to teach research methods to Ph.D. candidates and to conduct empirical research on democratization and consumer behavior. This time, I encountered another problem that is about equivalent models. In analyzing social science data, there are always tens or even hundreds of model that can fit a same set of data very well. How to deal with these issues became a focus of mine work for a while, and led me to focus on research process rather than research results. I feel we may not be able to obtain the so-called correct model for any data, but still need to obtain a best available or just a final model for our predictive work. Therefore, to a methodologist, what is important is not about how to reach a final model for a research, but the process leading to final models. During this period, I encountered the work done by a few computer scientists including professor Judea Pearl who used powerful AI tools to assist deriving causal relationships out from statistical evidence, and I even spent more than 4 months as a consulting scientist in IBM research to work on this set of methods. Through a careful examination of their work, I concluded that full automation of social studying is impossible that led me back to my idea of ResearchMaps. And also, I think the pursue for "the" causal model of any data set may lead to nowhere.

From this line of work, I became more and more familiar with the newest IT development that can offer huge assistance to social research. But its power has not been fully recognized yet by the empirical social research community. When the newest IT development brings out unlimited opportunities, it also brings out great challenges like that of information overflow. Now, I see many of our social research problems like equivalent models are amplified by the newest IT technologies. However, I started to believe that the newest IT may be the only cure to our problems or at least will create a path for us. The newest IT Technogical developments can make many unimaginable possible. One of these lines of thinking led to some work as summarized in this 2005 presentation of mine.

Excited by the IT revolution, I decided to focus my work on developing IT tools and IT solutions to social research. As mentioned earlier, I do not think all the research can be automated by computing tools or AI systems. But, IT systems are the most powerful tools to break down all the barriers in social research, such as the qualitative vs. quantitative and the single discipline approach vs. interdisciplinary approach. With all the newest IT tools in your hand, the world is flat. With some IT tools, we may be able to manage reflexivity issue as well. As for the researches scientific and capable to generate actionable knowledge, the ones I am interested, I think interdisciplinary approach is necessary, but is difficult to implement under current research establishment. We need the IT tools to flat our academic research world. Here, one of the problems is that scholars use difference terminologies to describe the same techniques. To cure this problem, RM4Es is what needed, as I believe. Now, I am developing RM4Es and its related IT tools.

To deal with the issues of reflexivity and equivalent models, I believe that Rmaps is truly the solution proposed and it started to get recognition now. As reflected by the unavoidable reflexivity and the existence of equivalent models, the “correct model” of a data may not exist at all. The final model reached by a researcher actually depends on our assumptions and on our preferences together with some of our research working habits. I think it is okay for us to stop at our final model and use it for predictions. However, we do need to know how this “final model” is different from the other possible ones so that a research map should be constructed and used to document the paths leading to our model and hopefully other possible paths leading to other possible final models. With this research map, we may guide our future directions and current actions of research. To provide Rmaps for others, I am working hard now to further develop the Rmap framework and its related IT tools.

 

Copyright @ 2006-2009  the RM Institute