avalies -ufrgs avaliaÇÃo da educaÇÃo superior€¦ · 9 unb 10 2 12 89 13.5 10 ufv 8 3 11 41...
TRANSCRIPT
AVALIES -UFRGS
AVALIAÇÃO DA EDUCAÇÃO SUPERIOR
PAINEL 1:
RANQUEAMENTO DA EDUCAÇÃO SUPERIOR
Jorge A. Guimarães
Porto Alegre, 17de setembro de 2015
Se os rankings são uma doença o
benchmarking pode ser a cura?
Jamil Salmi
who prepares the rankings?
0
1
2
3
4
5
6
7
8
9
Asia & the Pacific
Eastern Europe & Central Asia
Latin America & Caribbean
Middle Eastern & North Africa
North America Sub-Saharan Africa
Western Europe
Government Agency
Independent Agency
Press / Media
ranking systems in 2014
Region National and International Ranking SystemNational and International Ranking System
Eastern Europe and Central Asia Albania (A), Bulgaria (A), Kazakhstan (A, B), Latvia ©, Poland (C),
Slovakia (B), Romania (B/C), Russia (B, IB), Ukraine (B/C)
East Asia and Pacific
Australia (B), China (B, C, IB), Hong Kong (C), Japan (B, C), Malaysia (A),
New Zealand (A), South Korea ( B, C), Taiwan (B, IB), Thailand (A),
Vietnam (A)
Latin America and the Caribbean Argentina (A), Brazil (A), Chile (C), Colombia (B), Mexico (B, C), Peru (B)
Middle East and North Africa Israel (C), Tunisia (A)
North America Canada (B, C, B/C), United States (C)
South Asia India (A, B/C), Pakistan (A)
Sub-Saharan Africa Kenya (A), Nigeria (A)
Western Europe
France (IB), Germany (B/C, C), Ireland (C), Italy (C), Netherlands (A, IB),
Portugal (C), Spain (B, C, IC), Sweden (C), Switzerland (B/C), United
Kingdom (A, B, IC)
top 50 universities (2013)
ARWU ARWU 20132013
THETHE
20122012--1313
JAPAN; 2 CANADA; 2
UK; 5
WESTERN EUROPE; 6
USA ; 35
JAPAN, 1 AUSTRALIA, 2
CANADA, 3
OTHER ASIA, 4
WESTERN EUROPE, 4
UK, 7
USA, 29
Anglo-Saxon bias
s
top 100 universities / 1 million
people
0,01
0,02
0,04
0,05
0,06
0,09
0,11
0,14
0,16
0,18
0,18
0,20
0,29
0,31
0,36
0,38
0,52
0,00 0,10 0,20 0,30 0,40 0,50 0,60
Russia
Japan
Australia
Germany
France
Belgium
Canada
UK
USA
Netherlands
Finland
Norway
California
Sweden
Denmark
Israel
Switzerland
so should we just get rid of rankings?
1212
Autonomy
Academic Freedom
Students
Teaching Staff
Researchers
Leading-Edge
Research
Dynamic
Knowledge &
Technology
Transfer
Concentration
of Talent
Abundant
Resources Favorable
Governance
Leadership Team
Strategic Vision
Culture of Excellence
Public Budget Resources
Endowment Revenues
Tuition Fees
Research Grants
WPU Supportive
Regulatory
Framework
Top
Graduates
Characteristics of a Well-Performing University
Alignment of Key Factors
Source: Elaborated by Jamil Salmi
results
drivers of performance
2000
2014
clear goals
Building Minnesota’s Building Minnesota’s
worldworld--leading statusleading status
in the knowledge in the knowledge economy requires economy requires
setting goals for HE and setting goals for HE and measuring results.measuring results.
Governor Tim PawlentyGovernor Tim Pawlenty
benchmarking
assessing research excellence
Research Excellence
Quantity
Impact Quality
Internationalization of Brazilian Universities:
Challenges and Perspectives for
Institutions to Reach the Level of
World-Class University
Jorge A. Guimarães
Brasília, September 09, 2014
First Grade Challenges
for Brazilian Institutions to
Achieve WCU Stage
AUTONOMY
ACCOUNTABILITY
GOVERNANCE
Second Grade Challenges for Brazilian
Institutions to Achieve WCU Stage:
Top Ten Obstacles • Internationalization of Teaching and
Research;
• Reduce Informational Teach and Increase Formative Activities;
• Adopt International Curriculum;
• Offer Regular Courses in English and Other Languages;
• Apply Effective International Collaboration;
• Increase International Mobility of Students, Professors and Scientific Staff;
• Attract Foreign Students and Scientific Researchers;
• Offer Campus Residence;
• Stimulate International and Collaborative Publications;
• Offer Internships in Industry.
Challenges for Brazilian
Institutions to Achieve a World-
Class University Category:
How Many Brazilian Universities
Could Reach This Goal Today?
A Brief Study Based on Existing
Postgraduate Programs
CAPES RANKING - BRAZILIAN UNIVERSITIES
RANKING
NUMBER OF GRADUATE COURSES PER GRADE
University Grade 6 Grade 7 TOTAL 6 + 7 % CUMULATIVE %
1 USP 47 45 92 22.2 22.2
2 UFRJ 18 16 34 8.2 30.4
3 UNICAMP 17 16 33 8.2 38.6
4 UFRGS 19 13 32 7.7 46.3
5 UFMG 18 13 31 7.5 53.8
6 UNESP 14 3 17 4.1 57.9
7 UFSC 14 3 17 4.1 62.0
8 UNIFESP 6 6 12 2.9 64.9
9 UNB 10 2 12 2.9 67.8
10 UFV 8 3 11 2.7 70.5
11 PUC-RS 10 1 11 2.5 73.0
12 PUC-RJ 7 2 9 2.2 75.2
13 UFPE 8 ---- 8 1.9 77.1
14 UFPR 7 ---- 7 1.7 78.8
15 UERJ 4 3 7 1.7 80.5
16 UFSCAR 3 3 6 1.5 82.0
17 UFC 5 --- 5 1.2 83.0
18 UFF 3 1 4 1.0 84.0
19 UFLA 3 1 4 1.0 85.0
20 UFBA 3 1 4 1.0 86.0
24 IES 34 5 39 9.4 95.4
10 RESEARCH
INSTITUTES 11 8 19 4.6 100.0
TOTAL BRAZIL 270 145 415 11.8 100.0
CAPES RANKING - BRAZILIAN UNIVERSITIES
RANKING NUMBER OF GRADUATE COURSES PER GRADE
University Grade 6 Grade 7 TOTAL 6 + 7 PG % 6+7
1 USP 47 45 92 173 53.2
2 UFRJ 18 16 34 107 31.8
3 UNICAMP 17 16 33 71 46.5
4 UFRGS 19 13 32 84 38.0
5 UFMG 18 13 31 77 40.3
6 UNESP 14 3 17 105 16.2
7 UFSC 14 3 17 68 25.0
8 UNIFESP 6 6 12 53 22.6
9 UNB 10 2 12 89 13.5
10 UFV 8 3 11 41 26.8
11 PUC-RS 10 1 11 25 44.0
12 PUC-RJ 7 2 9 30 30.0
13 UFPE 8 ---- 8 75 10.7
14 UFPR 7 ---- 7 72 10.0
15 UERJ 4 3 7 53 13.2
16 UFSCAR 3 3 6 47 12.7
17 UFC 5 - 5 76 8.2
18 UFF 3 1 4 70 5,7
19 UFLA 3 1 4 29 13.8
20 UFBA 3 1 4 62 5.3
Outras 24 IES 34 5 39 869 4.5
10 RESEARCH
INSTITUTES 11 8 19 61 31.1
TOTAL BRAZIL 270 145 415 2276 18.2
BRAZILIAN UNIVERSITIES COMPARATIVE
RANKINGS
CAPES RANKING RUF RANKING (2012)
Ranking University Ranking University
1 USP 1 USP
2 UFRJ 2 UFRJ
3 UNICAMP 3 UFMG
4 UFRGS 4 UFRGS
5 UFMG 5 UNICAMP
6 UNESP 6 UNESP
7 UFSC 7 UFSC
8 UNIFESP 8 UNB
9 UNB 9 UFPR
10 UFV 10 UFPE
11 PUC-RS 11 UNIFESP
12 PUC-RJ 12 UFSCAR
13 UFPE 13 UERJ
14 UFPR 14 UFSM
15 UERJ 15 PUC-RJ
16 UFSCAR 16 UFC
17 UFC 17 UFBA
18 UFF 18 UFF
19 UFLA 19 PUC-RS
20 UFBA 20 UFV
22 UFSM 14 UFSM
19 UFLA 31 UFLA
BRAZILIAN UNIVERSITIES COMPARATIVE RANKINGS
CAPES RANKING RUF RANKING (2014)
Ranking University Ranking University
1 USP 1 USP
2 UFRJ 2 UFMG+1
3 UNICAMP 3 UFRJ-1
4 UFRGS 4 UFRGS
5 UFMG 5 UNICAMP
6 UNESP 6 UNESP
7 UFSC 7 UFSC
8 UNIFESP 8 UNB
9 UNB 9 UFPR
10 UFV 10 UFSCar+2
11 PUC-RS 11 UFPE-1
12 PUC-RJ 12 UNIFESP-1
13 UFPE 13 UFC+3
14 UFPR 14 UFBA+3
15 UERJ 15 UFSM-1
16 UFSCAR 16 UFF+2
17 UFC 17 UERJ-4
18 UFF 18 PUC-RS+1
19 UFLA 19 UFV+1
20 UFBA 20 PUC-RJ-5
21 UFLA 15 UFSM-1
22 UFSM 27 UFLA+4
Position Institution Courses Articles Position
1 USP 92 40,184 1
2 UFRJ 34 11,696 4
3 UNICAMP 33 12,845 3
4 UFRGS 32 10,325 5
5 UFMG 31 9,260 6
6 UNESP 17 14,563 2
7 UFSC 17 5,439 8
8 UNIFESP 12 8,001 7
9 UNB 12 4,323 12
10 UFV 11 4,680 10
11 PUC-RS 11 1,176 26
12 PUC-RJ 9 994 31
13 UFPE 8 4,498 11
14 UFPR 7 5,102 9
15 UERJ 7 3,498 17
16 UFSCar 6 4,048 13
17 UFC 5 4,013 14
18 UFF 4 3,631 16
19 UFLA 4 2,919 18
20 UFBA 4 2,832 20
Note: FIOCRUZ and EMBRAPA not included in the table,
produced 5,616 and 5,279 articles, respectively.
Sources: CAPES, GEOCAPES, September 2013
and INCITES, ISI, Thomson Reuters, September 2014.
Ranking CAPES: Courses 6 + 7
Ranking of Brazilian
Scientific Production
2009 - 2013
Fig. 2.
From William H. Press 2013, Science 342:817-822
Published by AAAS
outline
uses and abuses of rankings
from rankings to benchmarking
what benchmarking involves
a thin line between love and hate
disagreement with principle
criticism of methodology
boycotts
court actions (New Zealand, Holland)
create your own ranking
methodological problems • validity of criteria
• research bias & hard sciences
• peer evaluations
• English language publications
methodological problems •validity of criteria
• research bias& hard sciences
• peer evaluations
• English language publications
•validity of weights given to various criteria
• statistical robustness
methodological problems
(cont.)
• do all the indicators taken together actually measure quality?
• do the score differences reflect actual quality differences?
• are the rankings comparing the same types of institutions and programs?
danger of rankings
• changes guided by rankings criteria
• priority given to top students (equity concern) and/or
foreign students
• resource allocation (research)
• fraud in data presentation or survey participation,
payment of students
outline
uses and abuses of rankings
from rankings to benchmarking
positive features
system-wide focus
control for size
large range of indicators (multi-dimensional)
methodological problems
overall ranking with arbitrary weights
lack of statistical robustness
mix of results and determinants of these results
selection of indicators partly based on data availability,
not meaningfulness (equity)
mix of data from various years
outline
uses and abuses of rankings
from rankings to benchmarking
what benchmarking involves
what is benchmarking?
• comparing the performance of one’s tertiary
education system/institution to that of other
systems/institutions
• competitors
• good practices
proposing a theoretical
framework
• distinction between performance and health of system
• how good are the system’s actual outcomes?
• does it operate under conditions known to lead to high
performance?
• definition of outcomes / outputs / results
• identification of determinants and causality relationships
criteria for choosing
comparators?
• internal characteristics
• mission / philosophy / values
• types of programs (niche)
• research / teaching / learning philosophy
• characteristics of the environment
finding the data
• existing databases
• ad-hoc surveys
• field visits
• partnerships
• countries
• organizations
steps for benchmarking &
strategic planning
• what and how are we doing now?
• the right thing? is it relevant?
• is it effective and efficient (value for money)?
• what has changed in the environment?
• competition (domestic & international)
• resources
• regulatory framework
steps for benchmarking &
strategic planning
• where do we want to go?
• what sort of institution do we want to be (aims, values
and mission)?
• how do we get from where we are to where we want to
be?
• action plan (tasks, resources, structures)
• incentives
• strategic partnerships
accelerating factors
strategic planning and benchmarking
internationalization
being a niche institution and / or offering niche
programs
curriculum, pedagogical and managerial innovations