Myth: Impact Factor Measures Real Impact
Impact factor is the holy grail. A game that researchers play. Simply stated it means that greater the references to papers in other papers, the greater the impact. Supposing a Pakistani researcher's paper gets cited hundreds of times in ISI research journals of obscure names and places, what impact would it have created on the society here in Pakistan and on this nation. It is possible for a group of researchers to collude and using the you-scratch-my-back-and-I-scratch-yours approach to jack up the count of citations. I am told that some researchers are really playing this game well.
One of my friends who did his PhD from a prestigious US university told me about a course that he took in USA that he later dropped. The professor told him to write a paper that is so dense and complicated that it should not be understandable by any one. This should be done by including some complicated equations and matehmatical models and should revolve around terminology related to a particular field. The only restriction was to include a few references to the work of some professors in Europe. When those professors' papers were analyzed, it was found that the papers were equally dense, incomprehensible and even non-sensical. These works are all published in reputable journals.
There is an interesting analysis of computer generated non-sensical but completely grammatical and correct English papers that appear to be scholarly but totally gibberish which were sent for publication to various journals/conferences and which got accepted and were published. There is even a website that helps you generate scholarly research papers complete with tables, facts and figures that get accepted in various conferences. Even major publishers of online journals had published computer generated research papers.
I know of some scientists who have over 800 publications in a career spanning over 40 years. If you do the maths i.e. number of days in 40 years divided by the number of publications, the average number of days in which a publication was produced comes out to be around 18 days (= 40 x 365 / 800). What originality can there be in such creative work? Mind you majority of these publications would be in worthy impact factor journals.
I think there is a fundamental problem in this kind of measurement. You are measuring the instrument with itself. Fever can not be measured with fever of other people, it needs a device called thermometer that measures expansion of (say) Mercury. Speed of a car can not be measured by using speed of other cars, it needs a device called speedometer. Height of a plane in air can not be determined by comparing them with other planes in the air, it needs an instrument called altimeter that uses another quantity. The issue in impact factor measurment is measuring a quantity with itself. Hence, the number of publications have increased tremendously. There are now services available that would publish your journal and articles and would get your journal and articles a high impact factor if you pay some thousands of dollars.
One of my friends who did his PhD from a prestigious US university told me about a course that he took in USA that he later dropped. The professor told him to write a paper that is so dense and complicated that it should not be understandable by any one. This should be done by including some complicated equations and matehmatical models and should revolve around terminology related to a particular field. The only restriction was to include a few references to the work of some professors in Europe. When those professors' papers were analyzed, it was found that the papers were equally dense, incomprehensible and even non-sensical. These works are all published in reputable journals.
There is an interesting analysis of computer generated non-sensical but completely grammatical and correct English papers that appear to be scholarly but totally gibberish which were sent for publication to various journals/conferences and which got accepted and were published. There is even a website that helps you generate scholarly research papers complete with tables, facts and figures that get accepted in various conferences. Even major publishers of online journals had published computer generated research papers.
I know of some scientists who have over 800 publications in a career spanning over 40 years. If you do the maths i.e. number of days in 40 years divided by the number of publications, the average number of days in which a publication was produced comes out to be around 18 days (= 40 x 365 / 800). What originality can there be in such creative work? Mind you majority of these publications would be in worthy impact factor journals.
I think there is a fundamental problem in this kind of measurement. You are measuring the instrument with itself. Fever can not be measured with fever of other people, it needs a device called thermometer that measures expansion of (say) Mercury. Speed of a car can not be measured by using speed of other cars, it needs a device called speedometer. Height of a plane in air can not be determined by comparing them with other planes in the air, it needs an instrument called altimeter that uses another quantity. The issue in impact factor measurment is measuring a quantity with itself. Hence, the number of publications have increased tremendously. There are now services available that would publish your journal and articles and would get your journal and articles a high impact factor if you pay some thousands of dollars.
The more fundamental question that may be asked is can we use this impact factor measure to quantify the contribution of Steven Spielberg, Disney, Steve Jobs, Bill Gates, or Sam Walton (of Walmart fame) to the US society.
Can this impact factor quantify the positive contribution of Abdul Sattar Edhi, Dr Adeebul Hasan Rizvi, TCF, and so many others who are creating a sizeable impact on society.
Can this impact factor quantify the negative contributions of the dictators, bigots and war-mongers or exploiters of environment and plunderers of the financial industry in Pakistan. The impact factor is hopelessly out of sync with society and industry.
Can this impact factor quantify the negative contributions of the dictators, bigots and war-mongers or exploiters of environment and plunderers of the financial industry in Pakistan. The impact factor is hopelessly out of sync with society and industry.
The challenge for HEC in Pakistan was to define an impact factor that has relevance for Pakistan. It should have measured:
– Impact on the industry; new ventures, new products, new opportunities, new jobs, ..
– Social change, economic change, change movements
Instead of plagiarizing impact factor research and whole scale implementing it in Pakistan without thinking about its societal impact, we should have been prepared to do original research and should have researched to enhance the impact factor so that it can measure concrete changes brought about in our society. This should have been the role of higher education commission.
West is at a stage where the relevance of research to society has already been worked out and they can play this game. Pakistan is at a stage which has to ensure this societal impact as it spends its precious resources in research. Development of such an impact factor that quantifies contribution to our society is our need. It is not the need of West that has already transited from the stage, where we find ourselves at present.
What we need to do now is to develop indices that measure industry growth, job growth, poverty alleviation, health improvement and other such goals. Then we need to link the output of the university such as the contribution of the alumni to society, industry, and entrepreneurship. We should subtract from such positive contributions, all the negative contributions made by the alumni. There should be Hall of Fame for each university and then there should be a Hall of Shame. Let's measure the true impact. The impact on character of the graduates as reflected in their contributions to the society and industry.
People may argue that it is difficult, it may not be objective, it may not be measurable, what would be instrument, who would calibrate. But these are the challenges not the problems. Solving such challenges is the real impact factor research. If proponents of higher education would not do this, then who would!
– Impact on the industry; new ventures, new products, new opportunities, new jobs, ..
– Social change, economic change, change movements
Instead of plagiarizing impact factor research and whole scale implementing it in Pakistan without thinking about its societal impact, we should have been prepared to do original research and should have researched to enhance the impact factor so that it can measure concrete changes brought about in our society. This should have been the role of higher education commission.
West is at a stage where the relevance of research to society has already been worked out and they can play this game. Pakistan is at a stage which has to ensure this societal impact as it spends its precious resources in research. Development of such an impact factor that quantifies contribution to our society is our need. It is not the need of West that has already transited from the stage, where we find ourselves at present.
What we need to do now is to develop indices that measure industry growth, job growth, poverty alleviation, health improvement and other such goals. Then we need to link the output of the university such as the contribution of the alumni to society, industry, and entrepreneurship. We should subtract from such positive contributions, all the negative contributions made by the alumni. There should be Hall of Fame for each university and then there should be a Hall of Shame. Let's measure the true impact. The impact on character of the graduates as reflected in their contributions to the society and industry.
People may argue that it is difficult, it may not be objective, it may not be measurable, what would be instrument, who would calibrate. But these are the challenges not the problems. Solving such challenges is the real impact factor research. If proponents of higher education would not do this, then who would!
See Also:
- Motivation: Why PhD?
- What does it Mean to Have a PhD: Myths of Specialization and Departmental Expertise
- What is the Difference between MS/MPhil Research and PhD Research
Why PhD is Difficult:
- Why PhD is Difficult to Complete and Why there are so many ABDs and PhD Dropouts
- How Progress of Research is related to the Mood and Psychology of a PhD Student
Reading Research and Writing your Research
- How to Read a Research Paper and Extract Problem Statement and Thesis Statement
- How Literature Review of a PhD Dissertation Presents the State of the Art: Synthesis vs Listing
- What is a Problem Statement and its role in MS-PhD Research
- What is a Thesis Statement and its Role in PhD-MS Research
- What is meant by Rigor of PhD Research
- Conclusion vs Assumption in Research Writing- Flipping the Thread of Argument in your PhD Thesis
- PhD is about Pursuit of Excellence. Pursuit of Excellence vs Guzara: How to teach excellence through everyday examples
Qualitative Learning from a PhD
- Myth: Impact Factor Measures Real Impact
- Pursuit of Excellence vs Guzara: How to teach excellence through everyday examples
- Discerning the Forest from the Trees - The Insights from my PhD Supervisor JC Browne
- A Formula is Worth a Thousand Pictures: Dijkstra vs Buzan's Mind-Maps
- Fairness in Grading: A Lesson by the Great Dijkstra
- Lesser known dimensions of US Universities - Archives of history and literature
- Myth: We are backward because we Lag Behind in Science and Technology
- Myth: Mushrooming of HEIs in Pakistan
- Myth: Impact factor measures impact
- Myth: Increase in PhDs Increase Teaching Quality
- What is a PhD and What is its Definition
- Best Visualization of Bachelors vs Masters vs PhD that I have seen
- What is a Thesis Statement!
- 101 Tips for Completing your PhD Quickly
- Myths of Schooling and Education: Resources
- 5 Myths of Higher Education in Pakistan
Curriculum:
Sheheryar Mohsin Qureshi commented
ReplyDelete(Yesterday 2:05 AM)
Salam!
Being related with education sector as well as research, after reading it, i felt i had written these words. Very true for those who have finally decided to be a part of this damned "race of publications."
For the sake of information, i feel same is now happening in SCI indexed, SCI-e indexed and SSCI indexed journals. Or may be ISI index has now transformed into these indices by Thomson Reuters. Anyway, this is really thought provoking. Researchers have to decide the goal of research - number of publications or impact on society!
The Institute of Electrical and Electronic Engineers and Springer, a publisher of scientific journals and research, agreed to remove more than 120 fake studies following an investigation by Cyril Labbe of Joseph Fourier University.
ReplyDeleteLabbe spent two years analyzing research papers and discovered that more than 120 conference proceedings, as well as research papers attached to specific conferences, were published in 30 different journals, reports Nature. This occurred over the course of five years, from 2008 to 2013, and 16 studies were published in Springer journals and in journals published by the IEEE.
The full news is available at the below link:
http://www.ibtimes.com/fake-research-papers-how-did-more-120-gibberish-computer-generated-studies-get-published-1558725
Interesting. Very interesting
Delete