•
I.
Alex’s Journey of Methodology Study
When
I studied engineering in the Northwestern
Polytechnic University from 1978 to 1982, I was attracted to
social studies due to my strong wishes to participate in
China’s social reforms. Because of my strong social
participation intention and engineering background, I hoped all
good social researches to be scientific with results actionable.
At that moment, to me a college student who loves mathematics,
it seems to model social systems mathematically provides a lot
more challenges and fun than to model engineering systems.
Politically, I still trusted China’s political systems and
respected its leaders. However, I was puzzled by the many
horrible decisions China’s great leaders had made in the past.
Through a lot of reading, I came to conclude that rigorous and
scientific social studies were what China needed the most at
then. I believed that we should be able to construct scientific
social theories similar to physics and to run social management
as running engineering projects. Therefore, in 1981, I decided
to give up my engineering career and started my journey of
exploring methods of studying social systems.
As
guided by the thinking mentioned above, in 1983, I switched to
sociology and entered Beijing
University as a graduate student of sociology. At then, the
Beijing University sociology department favored empirical
research methods that my classmates and I got a lot of
assignments to learn and practice questionnaire survey and data
analysis. We even got to conduct surveys for the State
Council’s Economic Reform Committee, and to propose public
policy ideas. However, results produced from these surveys were
not satisfactory, as they never led to any good solutions but
only to some useful hints. I was a little disappointed by these
results, and thought they were caused by my lack of advanced
empirical research skills so that I needed to learn more.
Therefore,
in 1986, I entered a Ph.D. program at Stanford
University where I took many statistics and computing
courses besides sociological work. With the data I collected
from China, I tried some of the advanced data analysis tools I
learned. But, it came out to me these advanced tools were still
not enough to solve my problems that I started casting doubts on
the prevailing empirical research practice. One day in Stanford
Bookstore, I came cross a book Making It Count by
professor Stanley
Lieberson and immediately got attracted to it. In its
introduction, professor Lieberson wrote:
“There
were many failures before human successfully learned to fly.
After watching birds flap their wings, bold and adventurous
individuals built huge wing like structures, leaped off cliffs,
flapped their wings vigorously, and broke their necks. There are
principles of flight to be learned from watching the birds all
right, but the wrong analog had been drawn.”
Then,
he went on to argue that “in similar fashion, our empirical
approach to social behavior is based on an analog”, like
building wing like structures to fly. I did not agree with him
on many of his arguments made in the book. But the above analog
he made became something I never forget.
With
some doubts on the empirical social research practice, I still
moved on to learn more of empirical social research, but my
focus was on statistical techniques and computing tools for data
analysis since I thought they will always be useful anyway. I
took many advanced statistics courses and earned a M.S. of
statistical computing from Stanford. In practice, I participated
in the 1989 Beijing pro-democracy movement and got elected to be
the founding president of the Independent
Federation of Chinese Students and Scholars in USA after
returned from China. Partially due to this special experience, I
selected democratization as my main research topic and conducted
a few empirical researches on it. The result of this work led me
to produce a book Patterns
and Results of the Third Democratization Wave. However, my
research and my practice are separated from each other. My
research did not produce any meaningful guidance to my practice
that cast a lot more serious doubt on my believing in empirical
social research than ever. I felt this might be resulted from my
restrained to sociology and to quantitative analysis. In other
word, interdisciplinary approach and quantitative mixed with
qualitative methods may cure my problem. Here, I mean my way of
collecting data was restrained to the quantitative way and
sociological way. In general, I did not agree with the way most
scholars viewed the qualitative data. In terms of data, I think
that qualitative and quantitative are the same, and both can be
processed by computers.
Started
in 1993, I jumped into health economics first, then to the studies
of social networks and globalization, then to the fields of entrepreneurship
and international business, in order to be interdisciplinary.
All these studies, especially the study of entrepreneurial
process, are fascinating to me. However, more work led to more
doubts for me, especially when I had difficulty in dealing with
the subjectivity issue. The entrepreneurial process is
especially stimulating to me. At the beginning, entrepreneurs
often do some market research formally or informally. Then, they
act on what came out from this research. But the results out
from this process are often different from what suggested by the
early research. However, this does not prove the research is
wrong, but only to show the power of entrepreneurs’ will and
participation. This led me to some thoughts similar to George
Soros’s reflexivity. Also, the more work I do and the more
I got to see the huge difference between natural science and
social science, as I look for actionable knowledge out from
research. Definitely, additional human thinking is always needed
together with what research suggested to enable a good social
action.
From
2000 to 2005, I got a few opportunities to teach research
methods to Ph.D. candidates and to conduct empirical research on
democratization and consumer behavior. This time, I encountered
another problem that is about equivalent models. In analyzing
social science data, there are always tens or even hundreds of
model that can fit a same set of data very well. How to deal
with these issues became a focus of mine work for a while, and
led me to focus on research process rather than research
results. I feel we may not be able to obtain the so-called
correct model for any data, but still need to obtain a best
available or just a final model for our predictive work.
Therefore, to a methodologist, what is important is not about
how to reach a final model for a research, but the process
leading to final models. During this period, I encountered the
work done by a few computer scientists including professor
Judea Pearl who used powerful AI tools to assist deriving
causal relationships out from statistical evidence, and I even
spent more than 4 months as a consulting scientist in IBM
research to work on this set of methods. Through a careful
examination of their work, I concluded that full automation of
social studying is impossible that led me back to my idea of ResearchMaps.
And also, I think the pursue for "the" causal model of
any data set may lead to nowhere.
From
this line of work, I became more and more familiar with the
newest IT development that can offer huge assistance to social
research. But its power has not been fully recognized yet by the
empirical social research community. When the newest IT
development brings out unlimited opportunities, it also brings
out great challenges like that of information overflow. Now, I
see many of our social research problems like equivalent models
are amplified by the newest IT technologies. However, I started
to believe that the newest IT may be the only cure to our
problems or at least will create a path for us. The newest IT
Technogical developments can make many unimaginable possible.
One of these lines of thinking led to some work as summarized in
this
2005 presentation of mine.
Excited
by the IT revolution, I decided to focus my work on developing
IT tools and IT solutions to social research. As mentioned
earlier, I do not think all the research can be automated by
computing tools or AI systems. But, IT systems are the most
powerful tools to break down all the barriers in social
research, such as the qualitative vs. quantitative and the
single discipline approach vs. interdisciplinary approach. With
all the newest IT tools in your hand, the world
is flat. With some IT tools, we may be able to manage
reflexivity issue as well. As for the researches scientific and
capable to generate actionable knowledge, the ones I am
interested, I think interdisciplinary approach is necessary, but
is difficult
to implement under current research establishment. We need
the IT tools to flat our academic research world. Here, one of
the problems is that scholars use difference terminologies to
describe the same techniques. To cure this problem, RM4Es
is what needed, as I believe. Now, I am developing RM4Es and its
related IT tools.
To deal with the issues of reflexivity and equivalent
models, I believe that Rmaps
is truly the solution proposed and it started to get recognition
now. As reflected by the unavoidable reflexivity and the
existence of equivalent models, the “correct model” of a
data may not exist at all. The final model reached by a
researcher actually depends on our assumptions and on our
preferences together with some of our research working habits. I
think it is okay for us to stop at our final model and use it
for predictions. However, we do need to know how this “final
model” is different from the other possible ones so that a
research map should be constructed and used to document the
paths leading to our model and hopefully other possible paths
leading to other possible final models. With this research map,
we may guide our future directions and current actions of
research. To provide Rmaps for others, I am working hard now to
further develop the Rmap framework and its related IT tools.