{"id":3764,"date":"2018-09-12T05:27:00","date_gmt":"2018-09-12T05:27:00","guid":{"rendered":"https:\/\/cedos.org.ua\/researches\/derzhavni-koshty-dlia-akademii-pednauk-daty-ne-mozhna-zabraty-2\/"},"modified":"2021-03-15T14:13:03","modified_gmt":"2021-03-15T14:13:03","slug":"zvit-napnu","status":"publish","type":"researches","link":"https:\/\/cedos.org.ua\/en\/researches\/zvit-napnu\/","title":{"rendered":"Audit of the NAESU: conditions for public funding"},"content":{"rendered":"\n

Report on the Findings of the Study of the Conditions for the Efficient Use of Government Funding in the Research Activities of the NAESU<\/em><\/span><\/p>\n\n\n\n

\n
\n

Project Supervisor:<\/span> Yegor Stadny<\/p>\n

External Consultants:<\/span>
\n Yulia Bezvershenko, PhD (Candidate) in Physical and Mathematical Sciences, deputy head of the Council of Young Scientists of the National Academy of Sciences of Ukraine, vice president of the NGO Unia Scientifica,
\nSerhii Nazarovets, PhD (Candidate) in Social Communications, editor of E-prints in Library and Information Science and co-editor of the Directory of Open Access Journals (www.doaj.org).\n <\/p>\n

Readers:<\/span>
\n Oleksandra Antoniuk, PhD (Doctor) in Physical and Mathematical Sciences, deputy director of research activities at the Institute of Mathematics of the National Academy of Sciences,
\n Oleksandra Liashenko, PhD (Doctor) in Economic Sciences, professor, scientific secretary of the Institute of Strategic Research under the President of Ukraine\n <\/p>\n

\n Translation:<\/span> Roksolana Mashkova\n <\/p>\n

\n

\n The study was conducted as a part of the Think Tank Development Initiative for Ukraine, implemented by the International Renaissance Foundation in partnership with the Open Society Initiative for Europe (OSIFE), with financial support from the Embassy of Sweden in Ukraine.\n <\/p>

The opinions and positions articulated in this study belong to the authors and do not necessarily reflect the position of the Embassy of Sweden in Ukraine, the International Renaissance Foundation or the OSIFE.\n\n <\/p>

The authors are grateful to all participants of the interviews and the survey who found the time to answer our questions. All errors or inaccuracies, if any are found in the final version, are the sole responsibility of the authors.\n <\/p>

Any use of the materials contained in this study is allowed as long as the source is cited.\n\n <\/p>\n <\/div>\n <\/div><\/div>\n\n\n\n

\n
\n

Abbreviations<\/h2>\n NAESU<\/strong> \u2014 National Academy of Educational Sciences of Ukraine
\n NUS<\/strong> \u2014 New Ukrainian School
\n MES<\/strong> \u2014 Ministry of Education and Science
\n CSM<\/strong> \u2014 center for scientific methodology
\n SSPL<\/strong> \u2014 State Scientific and Pedagogical Library
\n HEI<\/strong> \u2014 higher education institution
\n NAQAE<\/strong> \u2014 National Agency for Quality Assurance in Education
\n IMCE<\/strong> \u2014 Institute for the Modernization of the Content of Education
\n UCEQA<\/strong> \u2014 Ukrainian Center for Educational Quality Assessment
\n IEA<\/strong> \u2014 Institute of Educational Analytics
\n IPPE<\/strong> \u2014 Institute of Postgraduate Pedagogical Education\n <\/div>\n
\n

Background Information<\/h2>\n

\n The National Academy of Educational Sciences of Ukraine is the central\n scientific institution in the field. The NAESU provides scientific and\n methodological support for the development of education, helps to design\n educational legislation and regulations, participates in the creation of\n the Concepts of the Development of Education. The NAESU consists of\n institutions which study education, pedagogy and psychology. The Academy\n was created by a presidential decree in 1992, and it received the National\n status in 2010.\n<\/p>\n

\n Since 1997 and until today, Vasyl Kremen (who was also the Minister of\n Education and Science in 2000-2005) has served as the President of the\n NAESU. The NAESU has 146 academics, and its institutions employ 1,430\n scientists, including 236 researchers with Doctor degrees and 547 with\n Candidate degrees (In the Ukrainian degree system, there are two postgraduate degrees: the junior degree is called the Candidate of Sciences (something like a \u201cJunior PhD\u201d), and the senior degree is called the Doctor of Sciences (something like a \u201cSenior PhD\u201d)). The employees of the institutions and the members of the\n Academy are working on 278 research topics in 42 research areas.\n<\/p>\n

\n The Academy has 14 research institutions: the Institute of Pedagogy, the H.\n S. Kostiuk Institute of Psychology, the Institute of Pedagogical Education\n and Adult Education, the Institute of the Problems of Upbringing, the\n Institute of Special Pedagogy, the Institute of Social and Political\n Psychology, the Institute of Higher Education, the Institute of\n Informational Technology and Means of Education, the Institute of\n Vocational-Technical Education, the Institute of Gifted Children, the State\n Higher Education Institution \u201cUniversity of Educational Management,\u201d the\n Ukrainian Center for the Scientific Methodology of Practical Psychology and\n Social Work, the Educational and Scientific Center for Vocational-Technical\n Training, the Sukhomlinsky State Scientific and Pedagogical Library of\n Ukraine.\n<\/p>\n

\n The NAESU is funded primarily from the State Budget of Ukraine. In 2018,\n UAH 211 million were allocated from the State Budget<\/a> to sustain the\n Academy.\n<\/p>\n <\/div>\n

\n

Introduction<\/h2>\n

\n Many countries constantly search for ways to spend public budget funds more\n efficiently. The higher the level of accountability and transparency of\n power, the more effort the government invests in searching for the optimal\n model of planning the budget spending.\n<\/p>\n

\n Since the Revolution of Dignity, the demand for efficient public spending\n has grown in Ukraine. First of all, the plan of government priorities to be\n implemented by 2020 has the increased effectiveness of budget programs\n listed as a component of the system of government finance management. This\n is the third year that the Ministry of Finance employs medium-term budget\n planning which, among other things, involves public reporting by the key\n funding managers about the results of their actions. Second, the demand for\n increased efficiency of public spending has grown due to the economic\n crisis and the deficit of budget funds.\n<\/p>\n

\n Simultaneously with the public finance management reform, many sectoral\n reforms have been launched in Ukraine. One of the key reforms is the\n education reform, whose every stage needs to be based on various scientific\n studies. The Ministry of Education and Science constantly seeks support\n from international partners and independent experts to implement the\n education reform \u2014 in particular, to carry out studies and design policies.\n Meanwhile, such studies in Ukraine are the domain of the National Academy\n of Educational Sciences of Ukraine (NAESU), which is one of the key public\n funding managers and receives many millions in budget funding every year to\n be spent on its activities.\n<\/p>\n

\n Today the system of public funding for the NAESU still does not require any\n evaluation of the efficiency of public money spending. Therefore, our study\n aims to determine if the procedures and practices of the planning and\n organization of research at the NAESU can ensure the efficient use of\n public funding in its scientific activities.\n<\/p>\n

\n The CEDOS Think Tank does not support any cuts in public spending on\n education. On the contrary, we believe that the government does not fund\n educational research sufficiently. At the same time, we believe that this\n research must be relevant and properly conducted. Because funding will be\n effective only when it is received by those research projects and\n institutions which can guarantee the proper conditions for the efficient\n use of the funding and, most importantly, ensure the high quality of the\n studies.\n<\/p>\n

\n We thank all the respondents who participated in this study, as well as our\n readers and external experts. The study authors are especially grateful to\n the NAESU representatives who participated in the discussion of the\n previous version of the report and provided their comments. In the final\n version, we have taken into account those comments which, in the authors\u2019\n opinion, could improve the report.\n<\/p>\n<\/div>\n

\n

Brief Overview of the Key Study Findings<\/h2>\n

\n The NAESU has formal procedures for determining the topscs of scientific\n research, as well as procedures for checking the adherence to the criteria\n of scientific quality. According to NAESU employees, these procedures also\n work in practice. At the same time, due to the lack of alternatives, there\n is usually no competition between different research proposals.\n<\/p>\n

\n The NAESU conducts almost three times as many fundamental studies as\n applied studies, and the fundamental studies receive more funding.\n Representatives of the NAESU explain this with the claim that the funding\n for fundamental studies is more protected.\n<\/p>\n

\n In addition, research at different NAESU institutions, even the studies\n whose topics are very similar, receive different amounts of funding. This\n is explained by the difference in the number and status of the employees\n who work on these studies, since the research budget is primarily based on\n the need to provide salaries to the employees. At the same time, the higher\n number of employees involved in the research does not guarantee that the\n work they do will be of a higher quality.\n<\/p>\n

\n The NAESU has its own clearly articulated procedures for checking the\n quality of studies during and after their realization. The procedures\n especially emphasize the indicators of implementation of research findings\n (particularly the publications), but these indicators do not necessarily\n point at the high quality of research. Meanwhile, the actual indicators of\n the quality of research appear in the NAESU institutions\u2019 annual reports\n only in 2017.\n<\/p>\n

\n The majority of the analyzed scientific papers published by the NAESU do\n not adhere to the basic requirements of quality, especially in terms of\n scientific literature analysis, methodology and the use of data. The\n disregard of papers which are not published in Russian and the small number\n of papers published abroad point at the isolation of Ukrainian educational\n research from global science. Only a small percentage of the NAESU\n scientific papers meet the international standards of quality.\n<\/p>\n

\n Some individual representatives of the NAESU participate in the development\n and implementation of the government educational policies in cooperation\n with the MES and the Verkhovna Rada. However, the NAESU studies are very\n rarely used in educational policy making. In addition, the functions of the\n NAESU in the field of methodological support for the educational process\n overlap with the IMCE functions, which is a sign of flaws in the relevant\n legislation.\n<\/p>\n

\n Despite the declared intent to intensify the collaboration with Ukrainian\n experts and with the public, the NAESU does not involve them in the\n planning of its activities. At the same time, some representatives of\n certain MES departments do participate in the planning of the collaboration\n with the NAESU, including the coordination of research topics.\n<\/p>\n

\n Respondents from the UCEQA and the IEA do not use the NAESU research,\n although they do work with its individual employees; meanwhile, IPPE\n representatives generally appreciate the collaboration with the NAESU and\n the scientific and methodological papers by the NAESU institutions.\n However, IPPE administrators note that these papers are not practical\n enough to be implemented in their work.\n<\/p>\n

\n Given the results of this study, there is no reason to believe that\n conditions for the efficient spending of budget funds existed in the NAESU\n during the studied period.\n<\/p>\n\n<\/div>\n

\n

Methodology<\/h2>\n

\n The study covered the NAESU activities in 2014-16. Therefore, all the\n documents and statistical data were about this period.\n <\/p>\n

\n The purpose of the study was to assess the conditions for the efficient\n spending of state budget funding in the research activities of the NAESU.\n In order to achieve this purpose, we set the following goals:\n <\/p>\n

    \n
  1. \n

    \n To describe the procedures of determining the topics of research at\n the NAESU and to evaluate the level of involvement of stakeholders\n in these procedures.\n <\/p>\n <\/li>\n

  2. \n

    \n To determine the tendencies in funding distribution within the\n NAESU.\n <\/p>\n <\/li>\n

  3. \n

    \n To describe the procedures and criteria of evaluation of the\n quality of research at the NAESU.\n <\/p>\n <\/li>\n

  4. \n

    \n To test papers published by NAESU employees for meeting the basic\n requirements for scientific papers.\n <\/p>\n <\/li>\n

  5. \n

    \n To determine the scope of practical implementation of the results\n of NAESU research activities, particularly in terms of government\n policy making in education and educational management.\n <\/p>\n <\/li>\n <\/ol>\n

    \n By the efficient use of government funding, we mean a transparent mechanism\n of distribution of government funding within the NAESU, which takes into\n account the sufficient scientific level of studies, the prioritization of\n research fields and topics, and the capacity of the study findings to be\n used by stakeholders.\n <\/p>\n

    \n The goals of the study were accomplished in three stages.\n <\/p>\n

    \n At the first stage, we carried out an armchair study to analyze the\n available sources: the NAESU regulations, reports and other statistical\n data (see Appendix 1). The majority of these documents were obtained from\n open sources (primarily from the official website of the NAESU), and some\n were received upon official requests. In addition, we tested NAESU papers\n for adherence to the basic requirements for scientific papers. In general,\n the NAESU scientific works include monographs, textbooks and handbooks,\n curricula and concepts, methodology handbooks and concepts, dictionaries\n and reference books, collections of scientific papers, and papers in\n sectoral journals. Naturally, the latter papers are the most numerous type\n of scientific publications at the NAESU, so we tested the articles for\n meeting the basic scientific requirements. The testing was carried out in\n the following steps:\n <\/p>\n

      \n
    1. \n

      \n We formed the list of all publications by three NAESU institutions\n included in the study (the institutions that were most frequently\n mentioned by independent education experts: the Institute of\n Pedagogy, the Institute of Higher Education and the Institute of\n Special Pedagogy) which were published in 2014-16 and were\n available online. Some of these publications were available from\n the NAESU electronic library, and some were openly accessible at\n the official websites of scientific journals or other websites.\n Purely journalistic papers were removed from the list. As a result,\n 663 articles remained in the sample (the list of articles<\/a>).\n <\/p>\n <\/li>\n

    2. \n

      \n The papers included in the sample were evaluated according to the\n following criteria:\n <\/p>\n <\/li>\n <\/ol>\n

        \n
      • \n

        \n Does the literature review include an analysis of studies on the\n paper\u2019s topic carried out by other authors in the past? (Yes\/No)\n (In this case, by \u201canalysis\u201d we mean an overview of the key results\n of studies published by other authors in the past.)\n <\/p>\n <\/li>\n

      • \n

        \n If the answer to the previous question is \u201cYes,\u201d then does the\n review contain an analysis of contemporary studies on the paper\u2019s\n topic? (Yes\/No) (By \u201ccontemporary studies,\u201d we mean the studies\n which were published: since 2009 for 2014 articles; since 2010 for\n 2015 articles; since 2011 for 2016 articles.)\n <\/p>\n <\/li>\n

      • \n

        \n If the answer to the first question is \u201cNo,\u201d then does the\n literature review mention the names of the authors who have studied\n the topic of the paper? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n If the answer to the previous question is \u201cYes,\u201d then does it\n mention contemporary papers? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n Does the review mention only (post-)Soviet authors? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n Does the paper explain the novelty of the study? (Yes\/No) (The very\n presence\/absence of the mention of novelty is recorded, without\n evaluating whether the study actually contributes anything new to\n science.)\n <\/p>\n <\/li>\n

      • \n

        \n Does the reference list include sources in foreign languages\n (except for Russian)? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n What is the percentage of contemporary publications in the\n reference list? (Indicate the percentage.)\n <\/p>\n <\/li>\n

      • \n

        \n Does the publication describe the methodology of the study?\n (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n Does the publication use any data processed by the author? (Yes\/No)\n (By \u201cdata processing,\u201d we mean the \u201cinspecting, cleansing,\n transforming, and modeling of factual information (such as\n measurements or statistics) with the goal of discovering useful\n information, suggesting conclusions, and supporting\n decision-making\u201d (according to the Wikipedia definition).)\n <\/p>\n <\/li>\n

      • \n

        \n Is the author a member of the editorial board (or any other\n administrative body) of the journal in which the paper is\n published? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n Does the journal in which the paper is published have a clearly\n defined subject field? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n If the answer to the previous question is \u201cYes,\u201d then does the\n paper\u2019s topic match the journal\u2019s subject? (Yes\/No)\n <\/p>\n <\/li>\n

      • \n

        \n Does the official website of the journal contain any information\n about how the manuscripts are reviewed for the journal? (Yes\/No)\n <\/p>\n <\/li>\n <\/ul>\n

        \n The evaluation was carried out by two experts: the CEDOS analyst Ihor\n Samokhin and the independent expert Serhii Nazarovets.\n <\/p>\n

        \n When we formulated the criteria for testing the scientific quality of\n publications, we based them on the rules of Western scientific journals in\n psychology and educational research. In particular, the American\n Educational Research Association (AERA), which publishes the highest-rated\n journals in education, has the following list of requirements for scholarly\n papers:\n <\/p>\n

          \n
        • \n

          \n development of a logical, evidence-based chain of reasoning;\n <\/p>\n <\/li>\n

        • \n

          \n methods appropriate to the questions posed;\n <\/p>\n <\/li>\n

        • \n

          \n observational or experimental designs and instruments that provide\n reliable and generalizable findings;\n <\/p>\n <\/li>\n

        • \n

          \n data and analysis adequate to support findings;\n <\/p>\n <\/li>\n

        • \n

          \n explication of procedures and results clearly and in detail,\n including specification of the population to which the findings can\n be generalized;\n <\/p>\n <\/li>\n

        • \n

          \n adherence to professional norms of peer review;\n <\/p>\n <\/li>\n

        • \n

          \n dissemination of findings to contribute to scientific knowledge;\n <\/p>\n <\/li>\n

        • \n

          \n access to data for reanalysis, replication, and the opportunity to\n build on findings.\n <\/p>\n <\/li>\n <\/ul>\n

          \n At the second stage, we conducted in-depth interviews with NAESU employees\n and key stakeholders about the questions which could not be answered at the\n previous stage. At the beginning of this stage, we interviewed\n representatives of ministries and other stakeholders. Based on their\n answers, we selected three NAESU institutions which were most frequently\n mentioned in conversations in order to conduct interviews with their\n employees. At the same time, during the preliminary consultations with the\n NAESU administration about the list of institutions, one of the NAESU vice\n presidents suggested that we drop the Institute of Psychology from the list\n and include the Institute of Special Pedagogy instead. This way, taking\n into account the respondents\u2019 mentions and the suggestion of the NAESU vice\n president, we formed the preliminary list of three institutions: the\n Institute of Higher Education, the Institute of Pedagogy and the Institute\n of Special Pedagogy. During the interviewing stage, respondents from one of\n the institutes refused to continue their participation in the study, so we\n additionally included the fourth institution, the Institute of\n Vocational-Technical Education (based on the same criterion of the\n frequency of mentions in interviews with the stakeholders).\n <\/p>\n

          \n The interviews were conducted face-to-face, by phone, by Skype, and one was\n conducted in the form of written replies. Before the beginning of the\n conversation, the interviewers informed the respondents about\n confidentiality and asked their permission to record the conversation.\n <\/p>\n

          \n To clarify the list of the necessary respondents among the representatives\n of the NAESU who are competent in our issues of interest, first we\n conducted exploratory interviews with employees of the NAESU Department of\n Research Organization.\n <\/p>\n

          \n In order to obtain the necessary information, we defined the following\n groups of respondents:\n <\/p>\n

            \n
          • \n

            \n representatives of NAESU institutions (Institute of Pedagogy,\n Institute of Special Pedagogy and Institute of Higher Education);\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the NAESU Department of Research Organization;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the NAESU Department of Finance and Economy;\n <\/p>\n <\/li>\n

          • \n

            \n members of the NAESU Presidium;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of a Department Bureau of choice;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the National Academy of Sciences of Ukraine;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the Ministry of Education and Science and its\n institutions (Ukrainian Center for Educational Quality Assessment,\n Institute of Educational Analytics, Institute of the Modernization\n of the Content of Education);\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the Ministry of Finance, Ministry of Social\n Policy, Ministry of Youth and Sports;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the Verkhovna Rada Committee for Science and\n Education;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of pedagogical HEIs;\n <\/p>\n <\/li>\n

          • \n

            \n representatives of the system of postgraduate pedagogical education\n (institutes of postgraduate pedagogical education);\n <\/p>\n <\/li>\n

          • \n

            \n independent experts in education (persons who are not affiliated\n with the structures mentioned above, but who are present in the\n educational discourse through their publications, interviews and\n comments on educational topics, their participation in working\n groups at government bodies, etc.).\n <\/p>\n <\/li>\n <\/ul>\n

            \n The interviews were conducted by CEDOS analysts Tetiana Zheriobkina, Iryna\n Kohut, Maria Kudelia and Yegor Stadny.\n <\/p>\n

            \n In total, we conducted 67 in-depth interviews between November 2017 and\n January 2018.\n <\/p>\n

            \n These included 13 interviews with representatives of the MES, its\n institutions and institutes of postgraduate pedagogical education; 2\n interviews with representatives of the Ministry of Finance and the Ministry\n of Youth and Sports; one interview with a representative of the National\n Academy of Sciences of Ukraine; 7 interviews with representatives of\n pedagogical HEIs; 18 interviews with independent experts in education; and\n 26 interviews with representatives of NAESU institutions and departments.\n <\/p>\n

            \n Therefore, the interviews with NAESU representatives made up the highest\n percentage of all the interviews, specifically 38.8 percent.\n <\/p>\n

            \n In the process of selecting respondents for the interviews, we took into\n account their level of involvement in the development and implementation of\n government policies in education, primarily in the spheres where the\n studied NAESU institutions operate.\n <\/p>\n

            \n The independent experts to be interviewed were selected based on their\n involvement in the development and implementation of the education reform,\n particularly in the work on the Law \u201cOn education,\u201d in the training of\n teachers for the NUS, in communication of the reform, etc.\n <\/p>\n

            \n From the institutions for postgraduate training of pedagogical workers, we\n selected the employees who played an active role in the reform, e.g.\n participated in the development of the State Standard for Primary\n Education, as well as in working groups for the reform of secondary\n education. One of the selected respondents did not find the time to meet\n for an interview during the interview stage of our research.\n <\/p>\n

            \n At the subordinate institutions of the MES (UCEQA, IEA, IMCE), we talked\n either to the institutes\u2019 heads and deputy heads, or to the employees who\n were suggested to us by the respondents from the MES.\n <\/p>\n

            \n At the MES, we interviewed the deputy ministers and department heads who\n work in the development and implementation of government policies in the\n fields in which the NAESU conducts research, as well as the representatives\n of the departments whom their bosses mentioned as the employees who\n maintain the closest contact with the NAESU (from the Department of the\n General Secondary and Preschool Education, the Department of Higher\n Education, the Department of Professional Education).\n <\/p>\n

            \n In order to study the practices of interaction between the NAESU\n institutions and the methodological departments and institutions for\n postgraduate pedagogical education, at the third stage of our research we\n sent invitations to regional institutes of postgraduate education to\n participate in an online survey. The survey covered the topics of the IPPE\n employees\u2019 contacts with NAESU institutions, events organized by these\n institutions, and their periodicals. The invitations were sent to the\n official emails of IPPEs; they contained the request to distribute the link\n to the survey among the institute employees. 39 respondents from 8 IPPEs\n answered the survey questions; most of them were from the Khmelnytskyi and\n Cherkasy Regions. The survey cannot be considered representative, but its\n results, in combination with the in-depth interviews with IPPE\n representatives, are used in this report with the caveat that they do not\n describe the experience of all the institutions of the system of\n postgraduate pedagogical education.\n <\/p>\n

            \n Limitations of the Study\n <\/h4>\n

            \n The study focuses on the analysis of the NAESU research during a limited\n period.<\/strong> The period is 2014-16, with the addition of 2013 and 2017 in the\n sections on funding. Therefore, the results cannot be extrapolated to any\n other time except for the indicated period and do not claim to describe the\n whole period of existence of the NAESU.\n <\/p>\n

            \n The sample of the NAESU respondents is not representative.<\/strong> The interviews\n were conducted only in 4 institutes (of 17), and each of them was\n represented by no more than 6 people (including their heads). Therefore,\n the answers they provided were mostly about the specific institutes and not\n about the situation at the NAESU in general.\n <\/p>\n

            \n The independent experts in education were selected based on their\n involvement in educational policy making.<\/strong> However, the respondents\u2019\n collaboration with the NAESU was not taken into account, so some of the\n respondents did not even have such experience. The sample is not\n representative and the interview answers do not reflect the whole picture\n of the NAESU\u2019s collaboration with external stakeholders who are experts in\n education.\n <\/p>\n

            \n The evaluation of adherence to the basic requirements for scholarly papers\n does not involve any meaningful criticism of the content of the views of\n NAESU representatives.<\/strong> Even if an article meets these criteria, this does\n not necessarily mean that it is of high quality; however, these criteria\n are the necessary conditions for meeting the high scholarly standards of\n research findings publishing and scientific discussion.\n <\/p>\n

            \n In order to evaluate the use of the NAESU work by IPPEs, we used two data\n collection methods: an online survey and in-depth interviews.<\/strong> This might be\n the reason why the evaluations based on the results of the online survey\n and on the in-depth interviews were so different. In the case of the\n in-depth interviews, the respondents knew the interviewers personally,\n which fostered trust and facilitated more honest answers.\n <\/p>\n

            \n Ethical limitations<\/strong>. We guaranteed absolute confidentiality to all\n respondents before the interviews and asked their permission to record the\n conversation. In order to ensure confidentiality, we do not use any\n gendered words in the report itself. Several respondents did not agree to\n the recording, so even though the conversations were conducted, their\n answers were not used in the study. In addition, several respondents\n behaved unethically with our interviewers, which forced us to terminate the\n cooperation and reduced the scope of information obtained during the study.\n <\/p>\n<\/div>\n<\/div>\n\n\n\n

            \n

            Part I. Planning and Selection of Research Projects at the NAESU<\/h2>\n
            \n

            Competition of Research Projects at the NAESU<\/h3>\n

            \n The main fields of research in which all the NAESU institutions work are\n approved by the General Assembly of the NAESU and contained in the document\n called The Priority Fields of Scientific Research of the NAES of Ukraine.\n It is regularly updated, and in November 2017 the NAESU approved its\n version for 2018-22.\n<\/p>\n

            \n Suggestions for priority fields of research are submitted to the\n Departments by the institutions\u2019 Scientific Councils, associate members and\n academicians. The guidelines for these suggestions are provided in the List\n of Priority Topics of Scientific Research and Technical Developments\n approved by a Decree of the Cabinet of Ministers of Ukraine. The research\n priorities are approved by the NAESU Presidium. According to some of the\n respondents, it happens after the suggestions are approved by the General\n Assembly of the NAESU. The listed fields are rather general, such as\n \u201cScientific and methodological basis for constructing the content of\n education at its different levels,\u201d or \u201cEnsuring the quality of and equal\n access to education in educational systems across the world.\u201d Based on\n these fields, the Presidium announces the annual competition of new\n research projects and determines the problems which must be targeted by the\n new research. Within the constraints of these problems, the NAESU\n scientists submit their projects for the competition.\n<\/p>\n

            \n The NAESU regulations, particularly the Regulations of the Planning and\n Implementation of Research Projects, determine that proposals about new\n topics and fields are to be developed by structural departments of NAESU\n research institutions. After that, they must be approved by the Scientific\n Councils of the corresponding research institutions and submitted for\n expertise to the relevant NAESU departments together with the calculated\n costs of implementation, which must be preliminarily approved by the NAESU\n Department of Finance and Economy. The basic criteria which are used to\n select the proposals are also defined. These include, for example, the\n relevance of the topic, its correspondence to the priority fields in which\n the institution works, the reasonability of the study in terms of the\n novelty of its scientific results, the research experience of the\n researchers who are going to work on the topic, the possibility of\n implementation of the study results. In addition, the list includes the\n criterion of having an approval letter from the potential consumer about\n the relevance of the particular research project for them.\n<\/p>\n

            \n The Department Bureau can involve special expert groups, competition\n commissions, experts among the NAESU employees or experienced education\n practitioners in the expert evaluation. After the evaluation, the results\n are approved by the NAESU Presidium, and the new research topics are\n included in the thematic plans which later serve as the basis for preparing\n the NAESU\u2019s budget proposals.\n<\/p>\n

            \n Some of the respondents also noted that the proposals about new fundamental\n topics must undergo expert evaluation at the National Academy of Sciences,\n while the proposals about practical topics must undergo evaluation at the\n MES. However, this procedure is not recorded in the Regulations described\n above.\n<\/p>\n

            \n According to the respondents, the actual procedure is the same as the\n nominal procedure described in the regulations:\n<\/p>\n

            \n \u201c…So they\u2019ve approved the topics at the Scientific Councils of the\n institutes, the subordinate institutions, and then they submit them to the\n Department Bureaus. The Department Bureau does the expert evaluation of the\n topics\u2026 Usually the experts are 2 or 3 people, they don\u2019t have to be\n members of the department, they can also be external experts, but the main\n criterion is that they are people who know about the field.\u201d<\/blockquote>\n<\/p>\n

            \n It is important to note that, according to the respondents, the Department\n Bureaus submit for the competition the problems which were proposed to them\n by the potential implementers themselves, namely the institutions, although\n the Bureaus can negotiate with the institutions about their proposals,\n change their wording or suggest clarifications. As a member of the NAESU\n Presidium noted, the proposals submitted for the competition usually have\n no rivals:\n<\/p>\n

            \n \u201cI\u2019ll tell you honestly, we rarely have two alternatives. \u2026 sometimes it\n can happen that we announce a problem, and two departments who, for\n example, completed their topics at the same time, see that both of them can\n work on this topic. And then we check if this topic can be combined, for\n example, for these two departments, to create something like a joint topic.\n We had such a case for biology and chemistry, for example, at the Institute\n of Pedagogy, I mean, they were finishing at the same time, the topic was\n natural sciences, they submitted two separate proposals, and then we\n thought, why don\u2019t we join them? The only thing is that the final product\n will be divided into two subjects, biology and chemistry. I mean, there are\n very few alternative proposals.\u201d<\/blockquote>\n<\/p>\n

            \n Therefore, we can assume that, given the lack of alternatives, it is enough\n to meet the basic requirements for the formal documentation of the proposal\n in order to win the competition of research projects.\n<\/p>\n

            \n So despite the fact that the procedure of determining research topics is\n defined as \u201ccompetition\u201d in the NAESU regulations, we can conclude that\n there is no actual competition between research proposals.\n<\/p>\n<\/div>\n

            \n

            Involvement of Stakeholders in Research Planning at the NAESU<\/h3>\n

            \n In addition to the practice of involving all employees of its institutions\n in the determination of priority fields and topics of research, the NAESU\n strategy for 2016-22 also proposes to expand the participation of\n stakeholders in this process. In particular, there is a plan to publish the\n draft lists of priorities and relevant problems at the NAESU website to\n generate wider discussion and take into account the suggestions of\n stakeholder organizations and persons, in particular civil society\n organizations, as well as professional associations.\n<\/p>\n

            \n However, all the independent educational experts who participated in the\n in-depth interviews noted that they were not involved in any formal\n discussions or other procedures aimed at defining the priority research\n fields for the NAESU or the relevant problems in education.\n<\/p>\n

            \n According to some of the respondents, this means that there is no interest\n on the part of the NAESU in any collaboration with NGOs and specific\n experts. At the same time, other respondents do not believe that the lack\n of formal discussions or consultations means that the NAESU is not\n interested and emphasize that informal connections and communication with\n the NAESU representatives also play an important role because they provide\n the opportunity to communicate one\u2019s opinion about the relevance of certain\n topics, one\u2019s interest in particular studies, etc.\n<\/p>\n

            \n \u201cFirst of all, there are no surveys. But it doesn\u2019t mean that there is\n nothing at all. Many real things happen during informal communication.\u201d<\/blockquote>\n\n<\/p>\n

            \n In general, the majority of respondents among independent experts in\n education are not familiar with the research and the main fields of the\n NAESU institutions. In their work, they mostly use foreign research.\n<\/p>\n

            \n\n \u201cAs a rule, I try to read international studies, articles in peer-reviewed\n Western journals and so on. To look up my topics of interest in Western\n journals. I have not encountered any publications by the Ukrainian Academy\n of Educational Sciences yet.\u201d<\/blockquote>\n\n<\/p>\n

            \n From the interview fragment quoted above, we can conclude that some of the\n people who work in education do not find the research they need among the\n studies conducted by the NAESU. Nevertheless, one of the experts, although\n he was not sure about the NAESU\u2019s research methodology, believed that the\n topics of its research are relevant and useful.\n<\/p>\n

            \n The fact that the surveyed experts in education are not very familiar with\n the NAESU\u2019s research can also be a sign of poor communication with the\n outside world. One of the respondents noted that this sphere needs to be\n improved:\n<\/p>\n

            \n\n \u201cBut they underestimate the meaning of public communication. They don\u2019t\n have a proper press office. When I read NAESU reports, I feel the urge to\n tear out the author\u2019s throat, as a professional journalist. Their writing\n is terrible, it\u2019s just impossible to write like that. I am not going to do\n their job for them and suggest what should be done, but I definitely see\n the result. The NAESU\u2019s image as an outdated, bureaucratic, inefficient\n structure is very strong. And sometimes, in my personal perception, it is\n dissonant with the actual things which I see thanks to my own paths of\n communication. But it shouldn\u2019t be like that. The image must be formed with\n different instruments.\u201d<\/blockquote>\n\n<\/p>\n

            \n The majority of the independent experts also noted that in addition to the\n lack of discussion of priority topics and relevant issues in education,\n they have not had any opportunity to collaborate with the NAESU. That is,\n they have not received any offers from the NAESU representatives to\n exchange data, consult about a certain study, etc. As for the NAESU\u2019s plans\n to increase the public\u2019s involvement in the discussions of priority\n research topics, it should be mentioned that some respondents from the\n NAESU noted that it is actually happening in some form, meaning that the\n institutions do monitor relevant problems, talk to experts and follow the\n social media.\n<\/p>\n

            \n\n \u201cThrough Facebook, through social media. We collect the relevance of\n problems\u2026 What practical workers, teachers, psychologists, practicing\n psychologists, school principals and, well, the general public are\n interested in.\u201d<\/blockquote>\n\n<\/p>\n

            \n However, some NAESU representatives noted in the interviews that they have\n not talked to any international partners, because some issues are relevant\n only for Ukraine.\n<\/p>\n

            \n\n \u201cWell, I\u2019ll tell you what, these are the problems which are relevant for\n Ukraine, for us, so, well, we have not involved [foreign partners or\n organizations in determining the priority fields]. This is what I\u2019ll say.\n We have not involved anyone, the only thing is that we used the experience\n of our partners, such as Poland, we used their experience, looked at the\n problems they have as EU members now, and we added them.\u201d<\/blockquote>\n\n<\/p>\n

            \n In addition to the independent experts, civil society representatives and\n international organizations, the MES can also indirectly influence the\n decisions about the NAESU research topics, because, according to its\n Statute, the NAESU carries out the independent scientific expert evaluation\n of drafts of planning documents (doctrines, concepts, strategies, etc.),\n educational innovations and educational literature, and by its own\n initiative develops suggestions about the foundations of educational\n policy, forecasts, informational and analytical materials, proposals,\n recommendations about the humanitarian development of the state and the\n improvement of the educational sphere.\n<\/p>\n

            \n Moreover, there is a plan of cooperation between the NAESU and the MES,\n which lists the research topics, the expected results and the methods of\n implementation. The plan is titled the Program of Collaboration between the\n Ministry of Education and Science of Ukraine and the National Academy of\n Educational Sciences of Ukraine for 2017-20. Such plans have been adopted\n every three years. Before they are adopted, they are sent to the MES for\n approval.\n<\/p>\n

            \n According to one of the respondents, these plans are first sent to the\n Minister of Education, then forwarded to deputy ministers who, in turn,\n forward them to the departments. This way, every thematic department\n participates in the approval of research topics included in these plans of\n collaboration.\n<\/p>\n

            \n However, representatives of some MES departments noted in the interviews\n that the approval is rather formal, while others said that they actively\n participated in the development of these plans of collaboration, suggested\n to introduce some changes to the research topics or to remove some of the\n topics from the plan. This difference may be explained by the different\n level of initiative at these MES departments or their different perceptions\n of these plans of collaboration, as well as by different perceptions of the\n possibility of getting involved in the actual NAESU planning.\n<\/p>\n

            \n As for the MES institutions (Ukrainian Center for Educational Quality\n Assessment, Institute of Educational Analytics, Institute for the\n Modernization of the Content of Education), the respondents who represented\n these institutions mostly noted that they had no influence on the plan of\n collaboration between the NAESU and the MES. One of the respondents noted\n that some NAESU representatives expressed wishes to find out which research\n topics can be relevant; however, these requests have not been official, but\n rather uttered in personal conversations. Representatives of some MES\n institutions also noted that their collaboration is most often aimed at\n coordination and support of the experiments conducted by the NAESU, and not\n at helping them determine the topics of studies or experiments.\n<\/p>\n

            \n Therefore, although the NAESU strategy for 2016-22 presupposes increased\n collaboration with foreign and Ukrainian experts and the public, the\n majority of experts involved in this study have no experience of\n collaboration with the NAESU or consulting for it. Meanwhile,\n representatives of MES departments have varying experiences of involvement\n in the planning of collaboration with the NAESU, including the experience\n of coordination in terms of research topics.\n<\/p>\n<\/div>\n

            \n

            \u0420\u043e\u0437\u043f\u043e\u0434\u0456\u043b \u0431\u044e\u0434\u0436\u0435\u0442\u043d\u043e\u0433\u043e \u0444\u0456\u043d\u0430\u043d\u0441\u0443\u0432\u0430\u043d\u043d\u044f \u043d\u0430\u0443\u043a\u043e\u0432\u0438\u0445 \u0434\u043e\u0441\u043b\u0456\u0434\u0436\u0435\u043d\u044c \u041d\u0410\u041f\u041d\u0423<\/h3>\n

            \n We sent public information requests to the NAESU and received the lists of\n topic plans of the Academy\u2019s institutions for 2013-17 (we used a longer period of time for this section than for the rest of the study in order to analyse the post-crisis tendencies in the NAESU funding). The lists contained\n the information about the type of research (fundamental or applied), the\n amount of funding allocated to the research project in a particular year,\n and the period of implementation (the year when it began and the year when\n it ended or is supposed to end).\n<\/p>\n

            \n In the last couple of years, the budget funding of research is mostly being\n reduced: while in 2013 all the NAESU research projects together received\n UAH 105 million, in 2016 they received only UAH 66 million. In 2017, the\n funding was increased to UAH 114 million, which is still much less than the\n 2013 amount after taking into account the inflation in 2013-16. Not all\n institutions experienced the cuts of research funding equally. The Table\n I.3.1 compares the institutions which received the highest amount of\n funding in 2013 and in 2017, and shows that in this period, the Institute\n of Pedagogy became \u201cmore important\u201d for the NAESU: in 2013, it received\n only one million hryvnias more than the Institute of Psychology, but in\n 2017 it received 6 million more.\n<\/p>\n

            \n Table I.3.1: The comparison of research funding for different NAESU\n institutions in 2013 and 2017.<\/em>\n<\/p>\n

            \n \n \n \n \n\n \n \n \n \n
            \n

            \n Institutions with the highest funding in 2013 (funding\n for research)\n <\/p>\n <\/td>\n

            		\n

            \n Institutions with the highest funding in 2017 (funding\n for research)\n <\/p>\n <\/td>\n <\/tr>\n<\/thead>\n

            \n

            \n Institute of Pedagogy \u2014 16 million\n <\/p>\n <\/td>\n

            		\n

            \n Institute of Pedagogy \u2014 22.5 million\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Psychology \u2014 15 million\n <\/p>\n <\/td>\n

            		\n

            \n Institute of Psychology \u2014 16.5 million\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Gifted Children \u2014 14,5 million\n <\/p>\n <\/td>\n

            		\n

            \n Sukhomlinsky SSPL \u2014 11 million\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Sukhomlinsky SSPL \u2014 9 million\n <\/p>\n <\/td>\n

            		\n

            \n Institute of Social and Political Psychology \u2014 10\n million\n <\/p>\n <\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n<\/div>\n

            \n Despite the major increase in funding in 2017 compared to 2016, the number\n of research projects did not increase (it is likely that the list of\n research projects was formed when there was no information about the\n increase yet). However, compared to 2013, all the institutions experienced\n a certain reduction of the number of research projects, and across the\n NAESU the number of research projects was reduced by about a quarter. In\n all institutions, the number of studies did not fall as much as the funding\n and the number of staff positions. For example, at the Institute of\n Pedagogy, the funding was reduced by 20% between 2013 and 2016, the number\n of staff positions was reduced by 21%, while the number of topics fell by\n only 12.5%. Therefore, the NAESU researchers could continue their research\n with less human and financial resources.\n<\/p>\n

            \n According to the annual NAESU reports in 2014-16, the number of fundamental\n research projects in 2014-16 was almost three times higher than the number\n of applied research projects. For example, in 2015, the NAESU conducted 90\n fundamental and 33 applied studies.\n<\/p>\n

            \n One of the representatives of the NAESU Presidium explained this by citing\n the special characteristics of budget demand and funding from the Ministry\n of Finance of Ukraine:\n<\/p>\n

            \n \u201cThere is one delicate aspect. If we include more fundamental studies in\n the budget, then they are protected.\u201d\n<\/p>\n

            \n In other words, for certain reasons, which remain unknown to us, it is\n easier to receive funding for fundamental research.\n<\/p>\n

            \n What can be said about the priority status of certain topics and the\n principles of distribution of research funding based on the money allocated\n for these topics? First of all, it should be noted that the sums vary\n widely: the average annual cost of research projects varies from UAH\n 166,000 to UAH 1,668,000 (The average annual cost is the average amount of funding allocated for one research project in one year). There are many more fundamental studies (423\n compared to 157 applied topics), but they are somewhat \u201ccheaper\u201d on\n average: 798,000 per year compared to 814,000 per year. There is also\n considerable inequality between institutions in terms of research funding.\n For example, the Institute of Gifted Children receives more than twice as\n much money per one research project every year than the Institute of the\n Problems of Upbringing.\n<\/p>\n

            \n Table I.3.2. The average annual cost of a research project at different\n NAESU institutions.<\/em>\n<\/p>\n

            \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
            \n

            \n Institution\n <\/p>\n <\/td>\n

            		\n

            \n Average annual funding allocated for one research\n project of the institution, UAH thousand\n <\/p>\n <\/td>\n <\/tr>\n<\/thead>\n

            \n

            \n Sukhomlinsky SSPL\n <\/p>\n <\/td>\n

            		\n

            \n 3,719\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Gifted Children\n <\/p>\n <\/td>\n

            		\n

            \n 1,206\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Informational Technology and Means of\n Education\n <\/p>\n <\/td>\n

            		\n

            \n 910\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Vocational-Technical Education\n <\/p>\n <\/td>\n

            		\n

            \n 909\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Higher Education\n <\/p>\n <\/td>\n

            		\n

            \n 908\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Social and Political Psychology\n <\/p>\n <\/td>\n

            		\n

            \n 757\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Pedagogical Education and Adult Education\n <\/p>\n <\/td>\n

            		\n

            \n 753\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n H. S. Kostiuk Institute of Psychology\n <\/p>\n <\/td>\n

            		\n

            \n 707\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Pedagogy\n <\/p>\n <\/td>\n

            		\n

            \n 700\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Ukrainian Center for the Scientific Methodology of\n Practical Psychology and Social Work\n <\/p>\n <\/td>\n

            		\n

            \n 694\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of Special Pedagogy\n <\/p>\n <\/td>\n

            		\n

            \n 682\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Institute of the Problems of Upbringing\n <\/p>\n <\/td>\n

            		\n

            \n 535\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n University of Educational Management\n <\/p>\n <\/td>\n

            		\n

            \n 247\n <\/p>\n <\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n<\/div>\n

            \n The research projects of the Sukhomlinsky SSPL turned out to be the most\n \u201cexpensive\u201d \u2014 they mostly cost the taxpayers 2 to 5 million hryvnias a\n year. But we can assume that these studies involved purchase of equipment\n or creation of databases and information systems: the topics of the most\n expensive research projects included \u201cThe organization and methodology of\n creating sectoral library information resources in the educational sector\n of Ukraine,\u201d \u201cThe scientific organization of informational activities of\n the SSPL sectoral library complex,\u201d etc.\n<\/p>\n

            \n The studies that stand out among other \u201cexpensive\u201d research projects are\n the applied studies of the Institute of Gifted Children dedicated to the\n diagnostics of giftedness and the support of gifted children \u2014 such topics\n received UAH 1.3-1.6 million a year. At the other institutions with large\n research budgets, the most expensive research projects were usually\n fundamental. By comparing different topics of studies that belong to the\n same type, we can distinguish the ones that received the highest funding\n and which apparently have the highest priority for the institution.\n<\/p>\n

            \n For example, the Institute of Psychology finished working on its\n fundamental study about \u201cThe psychological factors of self-determination of\n personality in the educational space\u201d in 2013, and took up the topic\n \u201cActive self-realization of personality in the educational space\u201d the next\n year, spending the average of over 1.4 million hryvnias on these topics\n every year. Other topics with seemingly similar titles received much less\n funding: \u201cThe self-projection of personality in the discursive space\u201d\n received the average of 637,000 a year, \u201cThe psychological and pedagogical\n foundations of the process of self-creation of personality\u201d received\n 592,000 a year. Meanwhile, the research projects titled \u201cThe educational\n environment as a factor of the development of a gifted personality\u201d\n (2012-14) and \u201cThe value determination of the development of a gifted\n personality\u201d (2015-17) cost 430,000-460,000, and the study on \u201cThe\n psychological patterns of realization of the constructive functions of\n ethnic culture in the informational society\u201d received only 300,000 per\n year. More money was received by the research projects which involved more\n people, sometimes dozens of people working on one topic. For example, the\n monograph on the constructive functions of ethnic culture was written by\n four people, and the report on the implementation of the study of \u201cThe\n psychological factors of self-determination of personality in the\n educational space\u201d had as many as 27 authors. It is rather difficult to\n understand why this particular research project requires dozens of authors\n and millions of hryvnias every year. Apparently, from the perspective of\n the Institute of Psychology, this is the topic that needs the most\n theoretical work (the report on the study shows that the work was mostly\n theoretical).\n<\/p>\n

            \n Fundamental studies on \u201ctheoretical\u201d topics receive the highest funding at\n the Institute of Pedagogy, too. In particular, the research project titled\n \u201cForming the content of specialized education in the unity of its\n invariable and variable components\u201d received 1.15 million hryvnias on\n average in 2015-17. However, some applied studies can also be unexpectedly\n \u201cexpensive.\u201d For example, the \u201cScientific and methodological support of the\n teaching of Ukrainian Geography in middle and high school\u201d received the\n average of one million hryvnias every year, although the \u201cScientific and\n methodological support of the teaching of Science in high school\u201d received\n only 550,000, and the \u201cScientific and methodological support of the\n intensified teaching of information technology in middle school\u201d received\n 420,000 (all of these studies were conducted in 2015-17). And in 2012-14,\n the research project titled \u201cScientific and methodological foundations of\n the formation and implementation of the updated content of education in\n Geography in middle school\u201d received 700,000 hryvnias a year. So, based on\n the thematic plans, one could conclude that the methodological research for\n Geography classes, on which the Institute\u2019s scientists constantly worked\n between 2012 and 2017, cost more than twice as much as the methodological\n research for Information Technology classes. It is hard to pinpoint the\n reason for such discrepancy, but the director of the Institute Oleh Topuzov\n is an author of handbooks and textbooks in Geography.\n<\/p>\n

            \n It is equally difficult to understand the principles of determining the\n priority of a particular research project in smaller institutions where\n most studies are methodologically similar. For example, the Institute of\n Higher Education has theoretical research projects titled \u201cThe phenomenon\n of the University in the context of the knowledge society\u201d and \u201cThe\n autonomy and governance in higher education.\u201d The first cost one million\n hryvnias every year on average, and the second cost 450,000. From the\n Institute\u2019s perspective, the first topic is probably much wider and\n therefore requires twice as much human and financial resources.\n<\/p>\n

            \n We did not analyse the budgets for each research project in detail (because\n of the large number of topics). Obviously, the brief analysis of the\n general distribution does not give us any idea about the factors that\n affect the cost of a certain topic. The NAESU representatives themselves\n explain that the amount of funding depends on the number and rank of the\n researchers who are involved in the study: the more PhDs, assistant\n professors and professors are involved in a research project, the more\n funding it requires. This explanation is logical, but it raises a number of\n questions. Why has this particular principle been used to determine the\n cost of research? And why are the employees so unequally distributed\n between some very similar topics? In addition, this explanation confirms\n the thesis of some critics of state scientific institutions that their\n research funding is actually the funding for their staff\u2019s salaries, and\n that the competitive selection of research projects exists only on paper.\n<\/p>\n<\/div>\n\n\n\n\n

            \n

            Part II. Quality Assurance of Research Activities at the NAESU\n<\/h2>\n
            \n

            Procedures and Criteria of Evaluation of the Quality of Research Activities at the NAESU<\/h3>\n

            \n According to the NAESU Regulations of the planning and implementation of\n research, in order to start a new research project at the NAESU, you need\n to participate in a competition which evaluates the relevance and\n usefulness of the suggested topic, the possibility of practical\n implementation of study results, and the authors\u2019 previous research\n experience. Each study that wins the competition must be accountable and\n regularly monitored, and fundamental research projects are also required to\n undergo state expert evaluation to determine their scientific level. The\n Regulations mention that any study can be terminated if it is not\n implemented on the proper level, although the criteria of the proper level\n are not articulated. The decision to terminate a research project is made\n by the head of the institution with the department\u2019s approval. The authors\n must report on the results of every stage of their research project, and\n after they finish the study, they must produce the final annotated report.\n The intermediate research findings are also recorded in the annual reports\n of the institution.\n<\/p>\n

            \n So the research projects at the NAESU are, according to the Regulations,\n regularly evaluated through reporting. At the same time, each project must\n meet the requirements indicated in its agreement or terms of reference. The\n finished studies are also evaluated based on the reporting documents\n submitted by their authors. These documents may include, in addition to the\n scientific report itself, any kind of study materials, electronic\n resources, etc. The authors may also attach any MES certificates (if they\n have created any pedagogical products), agreements about the implementation\n of research findings, patents or copyright certificates to their reports.\n<\/p>\n

            \n Each research project must undergo an expert review, but the person\n responsible for the review is the supervisor of the research project, which\n may cause a conflict of interest. In the interviews, NAESU representatives\n noted that there is both internal and external review process, and both\n elements are mandatory. However, the Regulations do not state this\n requirement. Reviews, reports and all the other materials are submitted by\n the supervisor to the Scientific Council of the institution, which, based\n on the materials and the oral report, decides whether the research project\n has been implemented and whether its results should be recommended for\n publishing. If the Council decides that the research project has not been\n implemented, the topic supervisor or the implementer of the research can\n request independent expert evaluation. If the expert evaluation also\n confirms the negative assessment of the study, the employment contract with\n its author can be terminated early.\n<\/p>\n

            \n The manuscripts written about the studies undergo additional expert\n evaluation which serves as a basis for the decision to include them in the\n annual printing plan. The evaluation is carried out by the department\u2019s\n expert council, a permanent advisory body whose members are department\n employees; however, external researchers can also be involved in the expert\n evaluation. According to the Regulations of the Procedure of Planning and\n Control Over the Implementation of Research Projects, certain topics can\n also be considered by the Presidium. \u201cThe most important scientific results\n of research by the subordinate institutions are heard by the Presidium of\n the NAES of Ukraine.\u201d\n<\/p>\n

            \n In the public reports of the NAESU published before 2017 there is little\n information about the quality of research. Many study results are labelled\n \u201csignificant\u201d without explaining where this conclusion comes from. The\n chapter on the \u201cImplementation of Research Findings\u201d contains mostly\n statistical data about the number of analytical materials and publications\n based on the studies, as well as the number of educational institutions and\n government bodies which \u201cimplemented\u201d these publications (it is not clear\n from the reports if schools actually do implement the methodological\n developments, or if they just receive printed copies of the publications).\n The NAESU also publishes collections of \u201cScientific Developments by the\n NAESU, recommended for implementation,\u201d which contain lists of publications\n \u2014 both scientific and methodological \u2014 published at the NAESU. Nearly every\n publication is accompanied by an explanation how its implementation will\n promote something. The attention to the implementation of research findings\n demonstrates that the NAESU pays attention, at least formally, to the\n practical significance of its developments.\n<\/p>\n

            \n One of the respondents from an IPPE noted that there is no evaluation of\n the effectiveness of the implementation of NAESU research, and this\n complicates public policy development, particularly the development of\n state standards:\n<\/p>\n

            \n \u201cI mean, obviously, there was some kind of research at some point, it did\n happen on a certain level. I mean, there is some kind of methodological\n concept, some educational strategies or something, which served as a basis\n for a certain approach, a certain line of textbooks and so on. But then\n this level of implementation became self-sufficient. And there was no\n deeper research, we rarely see any evaluation of the effectiveness of a\n certain approach or methodological toolkit which is based on that\n approach.\u201d<\/blockquote>\n<\/p>\n

            \n That is, even if a study was implemented by printing methodological\n recommendations or textbooks, the quality of these products and their use\n is not evaluated.\n<\/p>\n

            \n Under these circumstances, the implementation itself, without any\n evaluation of its results or impact, cannot definitively say anything about\n the quality of the NAESU scientific developments.\n<\/p>\n

            \n Since 2017, the NAESU reports have included indicators of research quality\n (in particular, they were listed in the report on 2016). These indicators\n include \u201cthe average citation index of scientists,\u201d \u201cthe average impact\n factor of scientific publications\u201d and \u201cthe number of NAESU journals which\n are included in citation databases,\u201d \u201cthe number of publications in foreign\n journals,\u201d etc. However, neither the reports nor the NAESU\u2019s Strategy of\n Development for 2016-22, which introduced these indicators, explain their\n meaning in more detail \u2014 for example, they do not name the citation index\n they use and the citation databases they mean (not all databases have\n strict criteria to ensure that they index only peer-reviewed scientific\n journals). Nevertheless, these indicators mark significant progress\n compared to the older reports.\n<\/p>\n <\/div>\n

            \n

            Evaluation of Publications of the NAESU Employees in Terms of Meeting the Basic Requirements for Scientific Articles<\/h3>\n

            \n The evaluation of whether the papers published by three NAESU institutes\n meet the basic requirements for scientific papers (see Methodology)\n produced slightly different results when carried out by the two experts,\n because they used different approaches. In the case of the criterion \u201cDoes\n the literature review include an analysis of studies on the paper\u2019s topic\n carried out by other authors in the past?\u201d, the CEDOS expert took into\n account that an overview article can contain analysis of previous work in\n the main body of the article, so it may satisfy this criterion, even if\n there is no analysis in the chapter specifically dedicated to the analysis\n of latest research. In addition, the CEDOS expert decided that historical\n papers which use primary sources (such as archival materials) satisfy the\n criterion \u201cDoes the publication use any data processed by the author?\u201d The\n independent expert had a narrower interpretation of these two criteria.\n However, he interpreted the criterion \u201cDoes the paper explain the novelty\n of the study?\u201d more widely, so in his analysis, more NAESU articles meet\n this criterion. The independent expert was also more strict about the\n question \u201cDoes the journal in which the paper is published have a clearly\n defined subject field?\u201d, because he specializes in scientific publishing.\n<\/p>\n

            \n The results of the analyses by both experts for all the criteria are listed\n in the Table below. When the criterion is conditional (starts with the word\n \u201cIf\u201d), then the table shows two values: the fraction among the papers that\n satisfy the condition and the fraction among all the analyzed papers.\n<\/p>\n

            \n Because of the different approaches and the impossibility of studying all\n questions in detail (for example, it is not always easy to find websites of\n Ukrainian journals), the Table sometimes demonstrates significant\n discrepancies. However, in the criteria of novelty, methodology and data\n usage the differences between experts are insignificant. As the final grade\n for all the criteria, we picked the one which is more favorable for the\n NAESU (emphasized with bold font in the table).\n<\/p>\n

            \n Table I.2.1. Comparison of the results of evaluation of whether papers by\n NAESU researchers meet the basic requirements for scientific papers.<\/em>\n<\/p>\n

            \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
            \n

            \n Criterion\n <\/p>\n <\/td>\n

            		\n

            \n Fraction of papers which satisfy the criterion,\n according to the CEDOS expert\n <\/p>\n <\/td>\n

            		\n

            \n Fraction of papers which satisfy the criterion,\n according to the independent expert\n <\/p>\n <\/td>\n <\/tr>\n<\/thead>\n

            \n

            \n Does the literature review include an analysis of\n studies on the paper\u2019s topic carried out by other\n authors in the past?\n <\/p>\n <\/td>\n

            		\n

            \n 28.5%\n <\/p>\n <\/td>\n

            		\n

            \n 9.2%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n If the answer to the previous question is \u201cYes,\u201d then\n does the review contain an analysis of contemporary\n studies on the paper\u2019s topic?\n <\/p>\n <\/td>\n

            		\n

            \n 60.7% (17.4% of all articles)\n <\/p>\n <\/td>\n

            		\n

            \n 77.6% (7% of all articles)\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n If the answer to the first question is \u201cNo,\u201d then does\n the literature review mention the names of the authors\n who have studied the topic of the paper?\n <\/p>\n <\/td>\n

            		\n

            \n 75.2% (52.7% of all articles)\n <\/p>\n <\/td>\n

            		\n

            \n 91,9% (83,5% of all articles)\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n If the answer to the previous question is \u201cYes,\u201d then\n does it mention contemporary papers?\n <\/p>\n <\/td>\n

            		\n

            \n 57.5% (30.5% of all articles)\n <\/p>\n <\/td>\n

            		\n

            \n 74.7% (62.9% of all articles)\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the review mention only works by (post-)Soviet\n authors?\n <\/p>\n <\/td>\n

            		\n

            \n 62.3% (51.2% of all articles)\n <\/p>\n <\/td>\n

            		\n

            \n 74.4% (73.3% of all articles)\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the paper explain the novelty of the study?\n <\/p>\n <\/td>\n

            		\n

            \n 4.4%\n <\/p>\n <\/td>\n

            		\n

            \n 10.8%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the reference list include sources in foreign\n languages (except for Russian)?\n <\/p>\n <\/td>\n

            		\n

            \n 21.9%\n <\/p>\n <\/td>\n

            		\n

            \n 23.6%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the publication describe the methodology of the\n study?\n <\/p>\n <\/td>\n

            		\n

            \n 8%\n <\/p>\n <\/td>\n

            		\n

            \n 7.4%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the publication use any data processed by the\n author?\n <\/p>\n <\/td>\n

            		\n

            \n 17.1%\n <\/p>\n <\/td>\n

            		\n

            \n 11.7%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Is the author a member of the editorial board (or any\n other administrative body) of the journal in which the\n paper is published?\n <\/p>\n <\/td>\n

            		\n

            \n 16.9%\n <\/p>\n <\/td>\n

            		\n

            \n 15.7%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the journal in which the paper is published have a\n clearly defined subject field?\n <\/p>\n <\/td>\n

            		\n

            \n 83%\n <\/p>\n <\/td>\n

            		\n

            \n 27.9%\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n If the answer to the previous question is \u201cYes,\u201d then\n does the paper\u2019s topic match the journal\u2019s subject?\n <\/p>\n <\/td>\n

            		\n

            \n 93.4% (73% of all articles)\n <\/p>\n <\/td>\n

            		\n

            \n 80.2% (21.9% of all articles)\n <\/p>\n <\/td>\n <\/tr>\n

            \n

            \n Does the official website of the journal contain any\n information about how the manuscripts are reviewed for\n the journal?\n <\/p>\n <\/td>\n

            		\n

            \n 39.1%\n <\/p>\n <\/td>\n

            		\n

            \n 31%\n <\/p>\n <\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n<\/div>\n

            \n Therefore, based on the evaluation of the NAESU papers in terms of\n satisfying the scientific standards, we can make the following\n generalizations:\n<\/p>\n

              \n
            • \n

              \n Only 28.5% of the articles contain analysis of previous research on\n the topic, in any form. Although most papers have a chapter\n dedicated to the analysis of recent research, this chapter usually\n contains only a list of names of the scientists who have worked on\n the topic, without references to the relevant work by these\n scientists. Such \u201canalysis\u201d does not let the reader know if the\n article\u2019s author even read other papers on his or her topic. Among\n the names listed in such analyses, there are usually many other\n NAESU researchers, while mentions of contemporary foreign authors\n are extremely rare.\n <\/p>\n <\/li>\n

            • \n

              \n 78% of the literature analyses and 75% of the lists of names (which\n are used as substitutes for literature analyses) mention\n contemporary authors. This percentage is logical, given the high\n number of the authors\u2019 contemporary NAESU colleagues in the lists\n of names.\n <\/p>\n <\/li>\n

            • \n

              \n 62% of the literature analyses and lists of names mention only\n Soviet and post-Soviet authors. In many other cases Western\n researchers are represented by only two or three names of the old\n masters of psychology and pedagogy (Montessori, Piaget, etc.). The\n analyzed articles allow us to understand that even though 27 years\n have passed since Ukraine gained independence, Ukrainian\n educational science still rarely remembers that foreign educational\n research exists. The exceptions are mostly papers written at the\n Institute of Higher Education and at the Department of Comparative\n Pedagogy of the Institute of Pedagogy.\n <\/p>\n <\/li>\n

            • \n

              \n Only 23.6% of all papers contain mentions of literature which is\n neither in Ukrainian nor in Russian. Even the papers written in English often mention exclusively post-Soviet literature written in Ukrainian or Russian.\n <\/p>\n <\/li>\n

            • \n

              \n Only 10.8% of the papers explain what the novelty of the material\n is. This fact in and of itself does not mean that 90% of the\n articles do not carry any novelty. However, it means that the NAESU\n researchers usually are not instructed to produce qualitatively new\n scientific knowledge and not encouraged to do it by the journals\u2019\n editorial policies. The chapters on \u201cformulating the purpose of the\n article\u201d in many papers contain phrases such as \u201cto outline the\n problem\u201d or \u201cto clarify the concept,\u201d without any intention to\n contribute anything to scientific knowledge about education. Thus,\n the majority of papers resemble student overviews rather than\n scientific articles. The small word count of the overwhelming\n majority of the papers (they are rarely longer than 10 pages) and\n the short reference list (rarely more than 15 points) only\n increases the resemblance to student papers.\n <\/p>\n <\/li>\n

            • \n

              \n Only 8% of the articles contain descriptions of the research\n method. If the methodology is explained, it allows the reader to\n understand how the scientist reached his or her conclusions\n described in the paper. The NAESU articles usually do not provide\n this explanation, although it is to be expected for such practical\n and empirical discipline as pedagogy.\n <\/p>\n <\/li>\n

            • \n

              \n 17% of the papers contain data which were processed and analyzed by\n the author to some extent. Almost a half of these papers are\n articles about the history of pedagogy, which use many old\n publications and archival materials. The NAESU papers rarely\n analyse any experiments or surveys conducted by authors, or any\n statistical data, although many Western journals (particularly in\n the field of education) do not even consider papers which do not\n mention any results of empirical research. Unfortunately, even\n those NAESU articles which described some results of original\n empirical studies often dedicated only two or three pages to those\n results, without any detailed analysis.\n <\/p>\n <\/li>\n

            • \n

              \n As for the journals, in 15.7% of all cases the paper\u2019s author is a\n member of the editorial board, which indicates a possible conflict\n of interest. The NAESU materials are mostly published in thematic\n pedagogical journals, but 17% of the articles are published in\n journals which have extremely wide topic range. This is especially\n the case for the papers in English, which were published, for\n example, in the Canadian Journal of Science, Education and Culture,\n the Modern Science journal, the Science Rise journal, and even the\n Oxford Journal of Scientific Research (no connection to the Oxford\n university). None of these journals are indexed in the Scopus\n database.\n <\/p>\n <\/li>\n

            • \n

              \n Only in 29% of all cases the experts managed to find any\n information about the review procedures at the journals (it is, of\n course, impossible to check whether these procedures are actually\n followed). However, the articles published in the non-peer-reviewed\n journals generally do not look fundamentally different than the\n articles in the peer-reviewed journals.\n <\/p>\n <\/li>\n<\/ul>\n

              \n The analysis covers three years, in which changes could have happened.\n However, the dynamics of these indicators over time does not allow us to\n see any significant tendencies of the scientific quality standards to\n improve or to deteriorate. The percentage of articles which mentioned only\n post-Soviet authors in their literature reviews fell from 66% in 2014 to\n 60% in 2015 and 2016; the methodology of research was described in 6.7% of\n all papers in 2014 and in 11% of all papers in 2016. These are the only\n even slightly noticeable shifts which we managed to record based on our\n analysis.\n<\/p>\n

              \n Therefore, the majority of the analyzed articles in some aspects fail to\n meet the high standards of scientific papers which could be published in\n peer-reviewed journals in Europe or North America. We must particularly\n emphasize such negative phenomena as the authors\u2019 tendency to list the\n names instead of analyzing the past research; the neglect of almost all\n Western research; as well as the virtual absence of empirical work with\n data.\n<\/p>\n<\/div>\n<\/div>\n\n\n\n\n

              \n

              Part III. Use of the Results of the NAESU Research Activities in the Development and Implementation of Policies in Education<\/h2>\n

              \n According to the NAESU Statute, the Academy interacts with government\n bodies, particularly with the Ministry of Education and Science, in order\n to do tasks which are determined by the government priorities in these\n spheres.\n<\/p>\n

              \n Among the tasks defined by the Statute is the independent scientific expert\n evaluation of drafts of prognostic documents (such as doctrines, concepts,\n strategies and so on), as well as the development, by the NAESU\u2019s own\n initiative, of suggestions about the foundations of educational policies,\n forecasts, informational and analytical materials, suggestions and\n recommendations about the humanitarian development of the state and about\n the improvement of the educational sphere. In addition, the NAESU, upon\n request from the President of Ukraine, the Verkhovna Rada of Ukraine, the\n Cabinet of Ministers of Ukraine, carries out the scientific expert\n evaluation of bills, draft government acts and programs.\n<\/p>\n

              \n The means of carrying out these tasks, which are listed in the Statute,\n include, in particular, participation in the development and implementation\n of government policies in education, science and technology.\n<\/p>\n

              \n According to the NAESU Report for 2012-16, the NAESU has developed and\n submitted to the Cabinet of Ministers, the Verkhovna Rada, the MES and\n other ministries, as well as to the institutions of the National Academy of\n Science, about 600 analytical materials, expert evaluations and suggestions\n for draft bills and regulations, including 238 suggestions for draft bills\n and more than 300 informational-analytical and informational-statistical\n materials for the Cabinet of Ministers of Ukraine. The NAESU\u2019s researchers\n prepared informational materials for and participated in 15 parliamentary\n hearings.\n<\/p>\n

              \n The analysis of the 2017-20 program of activities to be carried out\n together with the main NAESU\u2019s partner in the government, the Ministry of\n Education and Science, demonstrates that in the field of development and\n analysis of educational policies, the NAESU is going contribute to the\n development and improvement of legislation (by submitting suggestions for\n bills), to the development of conceptual and regulatory documents in\n education (concepts, strategies, action plans, regulations), to the\n research of the population\u2019s support for the government policies related to\n the education reform, and to the development of measures aimed to shape the\n positive public opinion about the introduction of the 12-year-long\n schooling and other education reforms.\n<\/p>\n

              \n The Academy\u2019s Strategy of Development for 2016-22 states that in order to\n implement the development priorities, the NAESU plans to integrate the\n innovative activities of its scientific institutions, education\n institutions and government institutions, and to take into account the\n suggestions of stakeholder organizations and persons while determining\n research topics.\n<\/p>\n

              \n The strategy also presupposes that NAESU scholars will have more initiative\n and participate more in the development and expert assessment of laws,\n decrees and regulations in education and science; that the NAESU will be\n effectively represented in expert and public councils and working groups at\n the MES and the Verkhovna Rada Committee on Science and Education, as well\n as in the NAQAE; that a procedure will be introduced for making joint\n decisions with the interested government institutions about the\n implementation of education reforms and modernization processes in\n education, pedagogy and psychology.\n<\/p>\n

              \n

              Use of the Results of the NAESU Research Activities in the Work of the Ministry of Education and Science and its Institutions<\/h3>\n

              \n The MES and its subordinate institutions involved in the development and\n implementation of government policies in education are some of the key\n partners of the NAESU and beneficiaries of its activities. Based on the\n results of in-depth interviews and the analysis of the cooperation plans,\n we can conclude that the NAESU most actively interacts with the MES itself\n and with the IMCE.\n <\/p>\n

              \n In-depth interviews with MES employees demonstrated that the experience of\n cooperation with the NAESU and of using the results of its research\n differed depending on the position and the sphere in which the respondents\n worked (most of all on the educational level in which they developed\n policies). NAESU workers are invited to participate in all the official\n working groups of the MES and its institutions, because the MES Regulations\n require it, but their actual contribution depends on the expert level of\n the particular person. The NAESU also carries out expert evaluation for the\n MES, particularly the evaluation of state educational standards.\n <\/p>\n

              \n There is a stable connection between the MES and NAESU institutions and\n their departments. The connection manifests either in informal personal\n contacts of MES employees who are familiar with the topics on which the\n NAESU institutions or individual NAESU researchers work and who talk\n directly to the specialists they need, or in official requests from the MES\n in the cases when fulfilling the request takes a lot of time, and the\n researchers need to insert it in their working plans. However, judging from\n the interviews, the first type of cooperation is more prevalent.\n <\/p>\n

              \n Some respondents from the MES (working with secondary, higher and inclusive\n education) said that they used NAESU research findings in their daily work.\n They emphasized that they used practical research which can be used in the\n development of methodological recommendations, study plans, analytical or\n informational notes; they also said they read NAESU periodicals \u2014 again, of\n methodological rather than scientific kind. The respondents working with\n other levels of education do not use NAESU research in their work.\n <\/p>\n

              \n Some respondents from the MES noted that they are familiar with the work of\n some NAESU researchers, but that the general level of research and the\n professional competence of the employees at NAESU institutions is not very\n high, and the results of the NAESU activities do not meet the respondents\u2019\n expectations of quality. In addition, the respondents noted that many\n results of the NAESU research activities are \u201ctheoretical\u201d and therefore\n inapplicable in their work, both because of their length and complicated\n writing style and because their format does not correspond to the needs of\n policy making in education. The MES representatives noted that the NAESU\n does not inform them about its work and does not sufficiently communicate\n the results of its research in a form which would be convenient to\n understand and use. However, another possible reason for the fact that MES\n representatives are not very familiar with the NAESU research is that the\n general practice of using research to make policy decisions in education\n has not been adopted in Ukraine.\n <\/p>\n

              \n In general, current decision making, according to one of the respondents,\n happens within the MES \u2014 NAESU \u2014 IMCE triangle. At each of these\n institutions, the employees who have similar areas of responsibility or\n research interests actually cooperate and interact. As for the higher-level\n policies (and most of all the drafting of bills), the NAESU participates in\n their making through its representatives in working groups, but these\n representatives are a rather small group of scientists with the highest\n status (academicians, Presidium members). However, as a MES representative\n noted,\n <\/p>\n

              \n \u201cWe make decisions based on experience, intuition, some kind of general\n understanding, general tendencies, rather than specific research\u2026\u201d\n <\/p>\n

              \n At the same time, the need for research which could serve as a basis for\n public policy making was also expressed by the respondents, who emphasized\n that such research must be practical rather than written \u201cfor the vacuum\u201d\n or just to be eligible for PhD degrees.\n <\/p>\n

              \n Among the subordinate and affiliated institutions of the MES, the IMCE is\n the most closely connected to the NAESU. For example, in the Program of\n Collaboration between the MES and the NAESU, the IMCE is mentioned as the\/a\n MES implementer for 72 activities out of 155 in 2017-20, and for 68\n activities out of 85 in 2014-16 (as the Institute of Innovative\n Technologies and Content of Education). The UCEQA and IEA are barely\n mentioned: the current Program mentions UCEQA only once.\n <\/p>\n

              \n All the respondents from the IMCE spoke about close collaboration with\n NAESU institutions, both formal and informal. The respondents use the NAESU\n research findings, such as journal papers, monographs, methodological\n developments etc., mostly of methodological rather than scientific kind\n (this is the most typical for the field of secondary education). They also\n attend events organized by NAESU institutions, accompany the NAESU\n experiments, involve the researchers in their activities (for example, as\n teachers for experts who evaluate drafts of school textbooks), as well as\n in scientific and methodological commissions and in the expert evaluation\n of their regulations.\n <\/p>\n

              \n It should be noted that the IMCE\u2019s and the NAESU\u2019s functions, both formal\n and actual, are generally similar. From the respondents\u2019 answers, one could\n conclude that the NAESU work which is the most relevant for the MES and the\n IMCE is the part in which the NAESU carries out the IMCE functions. The\n latter institution, according to its Statute, is supposed to provide\n scientific and scientific-methodological foundation for the modernization\n of the content of education by doing research and implementing its\n findings. However, in reality, the IMCE efforts are largely concentrated in\n the organizational-administrative plane, while the NAESU is an extension of\n this institution and takes over some of the work which is essentially the\n IMCE\u2019s responsibility (primarily the research about teaching methods and\n implementation of those methods, the development of curricula and\n textbooks). Below, we list some of the similar functions of both\n institutions:\n <\/p>\n \n

              \n \n \n \n \n \n \n \n \n \n
              \n

              \n IMCE\n <\/p>\n <\/td>\n

              		\n

              \n NAESU\n <\/p>\n <\/td>\n <\/tr>\n <\/thead>\n

              \n

              \n \u201cfundamental and practical research of issues in\n education, pedagogy, psychology and related sciences\u201d\n <\/p>\n <\/td>\n

              		\n

              \n \u201ccarries out fundamental and practical research and\n innovative development in education, pedagogy and\n psychology\u201d\n <\/p>\n <\/td>\n <\/tr>\n

              \n

              \n \u201corganization and implementation of\n scientific-methodological and psychological-pedagogical\n expert evaluation of curricula, educational literature,\n educational tools, and educational equipment of general\n and specific purpose\u201d\n <\/p>\n <\/td>\n

              		\n

              \n \u201cparticipates in independent scientific expert\n evaluation\u2026 of educational and methodological\n literature (except for the literature in higher\n education)…\u201d\n <\/p>\n

              \n \u201ccarries out the psychological expertise of educational\n standards, typical curricula, textbooks, methodological\n materials, etc.\u201d\n <\/p>\n <\/td>\n <\/tr>\n

              \n

              \n \u201cdevelopment of the standards of education and\n curricula, organization of their expert evaluation\u201d\n <\/p>\n <\/td>\n

              		\n

              \n \u201cparticipates in the development of teaching methods,\n standards of education, typical curricula, textbooks\u201d\n <\/p>\n <\/td>\n <\/tr>\n

              \n

              \n \u201cdevelopment and implementation of contemporary\n educational informational and innovative technology in\n education\u201d\n <\/p>\n <\/td>\n

              		\n

              \n \u201cdevelops and experimentally tests innovative models of\n education\u201d\n <\/p>\n <\/td>\n <\/tr>\n

              \n

              \n \u201cscientific and scientific-methodological support for\n socio-pedagogical work\u201d\n <\/p>\n <\/td>\n

              		\n

              \n \u201ctakes organizational, coordinative, and\n scientific-methodological measures to support practical\n psychology in education and social pedagogy\u201d\n <\/p>\n <\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n <\/div>\n

              \n The UCEQA and IEA virtually never use the NAESU work. The respondents from\n the UCEQA noted that they had not used the NAESU work results as such (that\n is, its publications, events, etc.), but that they use the expert knowledge\n of particular academicians and employees of the NAESU institutions as they\n carry out their tasks, both current (determining the threshold grades for\n the External Independent Evaluation) and for particular projects, most of\n all for working on the PISA study in Ukraine. The NAESU researchers\u2019\n contribution to the preparation of the study was highly appreciated by the\n respondents, but the NAESU research findings they were familiar with were\n deemed rather theoretical, while the UCEQA needs are more practical. One of\n the respondents used a metaphor to describe it, saying, \u201cWe are two\n different worlds.\u201d\n <\/p>\n

              \n The respondent from the IEA does not personally use the NAESU work, but his\n colleagues have worked with the materials of the TIMMS study, which was\n organized and whose findings were analyzed in Ukraine by the NAESU\n (however, the respondent noted that the report about the study has never\n been published). In addition, the IEA engages the NAESU expert knowledge by\n inviting NAESU representatives as working group members and by asking them\n to give expert conclusions about its research programs. The results of this\n cooperation were characterised as contradictory, partially positive and\n partially negative.\n <\/p>\n

              \n It should be noted that many respondents from the MES and its subordinate\n institutions, when talking about the collaboration with NAESU\n representatives by involving them in working groups, mostly mentioned the\n same names, emphasizing those individuals\u2019 specific contributions to policy\n making. Therefore, we can assume that there is a relatively small (compared\n to the general number of academicians, associate members and employees of\n NAESU institutions) group of scientists within the NAESU whose expert\n knowledge is in the highest demand and whose contribution to policy making\n is the most significant.\n <\/p>\n

              \n As for the use of NAESU work in the activities of other central executive\n government bodies (the Ministry of Social Policy, the Ministry of Finance\n and the Ministry of Family, Youth and Sports), the in-depth interviews with\n representatives of the two latter institutions showed that they do not use\n the NAESU work, while the Ministry of Social Policy refused to be\n interviewed because it does not collaborate with the NAESU.\n <\/p>\n <\/div>\n

              \n

              Use of the Results of the NAESU Research Activities in the Development of Legislation<\/h3>\n

              \n In the period under consideration, two core laws in the field of the\n NAESU\u2019s expertise were passed: the framework Law \u201cOn education\u201d and the Law\n \u201cOn science and scientific-technological activities.\u201d\n<\/p>\n

              \n Individual NAESU representatives were members of the working group that\n drafted the bill \u201cOn education,\u201d and they represented the highest level in\n the Academy\u2019s structure; that is, the working group included three\n Presidium members. They could have participated in the creation of the\n version of the bill which passed the first reading in the parliament. But\n it is impossible to isolate their contributions, except for the article on\n scientific and methodological support of education, which was supplemented\n before the first reading with a segment on the role and tasks of the NAESU.\n However, between the readings, organizations and institutions could submit\n their suggestions to the Parliamentary Committee on Education and Science,\n so at least we can analyze what the NAESU suggestions were about.\n<\/p>\n

              \n The NAESU institutions submitted a total of 71 suggestions and corrections\n to the draft bill before the second reading. 50 of them were made by the\n Institute of Higher Education, and the rest by six other institutions. A\n considerable fraction of the suggestions (10 out of 71) were about the\n definitions in Article 1, the law\u2019s glossary. A few others reflected the\n NAESU positions on the standardization of education and conveyed an\n understanding of educational standards specifically as government\n standards. In addition, the Institute of Higher Education suggested to\n include a provision that would determine that the state provides\n educational services in the predefined amount and has to define the cost of\n these services for each student, but only on the level of general secondary\n education \u2014 there were no similar suggestions for the articles on other\n educational levels. In addition, the Institute of Higher Education offered\n its own version of the chapter on the financial-economical relations in\n education.\n<\/p>\n

              \n Five of the NAESU suggestions were accepted. The working group members\n decided to accept the definition of autonomy of educational institutions\n and the list of the levels of education which were suggested by the NAESU,\n and to accept the suggestion to reject an idea about a separate law on\n higher professional education (it should be noted that other stakeholders\n who also submitted suggestions for the same article opposed the creation of\n this separate law, too).\n<\/p>\n

              \n The NAESU suggestions for the draft bill \u201cOn science and\n scientific-technical activities\u201d (10 in total) were exclusively about the\n status and rights of national sectoral academies of science (as well as the\n National Academy of Science of Ukraine). These were suggestions for Article\n 18 (National Sectoral Academies of Science) and Article 27 (Preparation of\n Scientific Staff and Their Further Training), as well as for the part of\n the Final and Transitional Provisions of the Law (paragraph 6,\n sub-paragraphs a-d) which were aimed to introduce changes in the Law \u201cOn\n higher education\u201d according to the regulations of Article 27. The accepted\n suggestions were about the non-profit status of research institutions of\n the national sectoral academies of science, the coordination of these\n institutions\u2019 activities by the Cabinet of Ministers without violating\n their autonomy, the possibility for the National and the national sectoral\n academies of science to found HEIs, as well as the possibility for their\n research institutions to train MAs, PhDs and Doctors of Science.\n<\/p>\n

              \n The absolute majority of the NAESU suggestions which were accepted in the\n final text of the bill were also submitted by the National Academy of\n Science and\/or articulated in the alternative bills 2244a-2, 2244a-3,\n 2244a-4 or in the suggestions of other participants of the working group\n that finalized the government\u2019s bill 2244a before the second reading.\n<\/p>\n<\/div>\n

              \n

              Use of the Results of the NAESU Research Activities in the Work of Methodology Departments and Educational Institutions<\/h3>\n

              \n According to the survey of the workers at regional institutions of the\n postgraduate pedagogical education system, the majority of them use the\n NAESU research findings in their work. 87% of the respondents participated\n in events organized by NAESU institutions (alone or in cooperation with\n other institutions) in 2014-17. Half of them participated in 1-3 events,\n and 30% in 4-6 events. 20 out of the 34 respondents who answered this\n question claim that these events are an important component of their work\n which improves the quality of their scientific and pedagogical activities,\n and 9 of them defined their role as an additional activity aimed to study a\n certain topic in-depth, which does not have any significant impact on their\n daily work. 77% of the respondents highly evaluated the relevance of the\n topics of the majority or a considerable percentage of these events. 70% of\n the respondents are definitely going to use the knowledge and skills they\n learned at these events in their work, and another 27% are probably going\n to do it. 70% have heard their colleagues speak about the NAESU events they\n attended, and these mentions were mostly positive in many or in nearly all\n cases. Judging from the responses we received, the experience of\n participation in the events organized by the NAESU is evaluated by IPPE\n workers as generally positive and useful.\n<\/p>\n

              \n 92% of the respondents have read printed materials (papers, monographs)\n written by employees of NAESU institutions or in collaboration with them in\n 2014-17. 44% of them defined what they had read as very useful and\n confirmed that they were going to use the knowledge they had learned,\n another 42% of them defined it as potentially useful, and another 11% found\n only particular useful aspects in what they read. 80% of the respondents\n read NAESU journals in the specified period, and 84% of them said that the\n majority or a considerable fraction of the materials published in these\n journals were of high quality and were interesting.\n<\/p>\n

              \n In-depth interviews with IPPE managers showed that some of the studies\n produced by the NAESU lack a practical aspect, a possibility to use the new\n knowledge in the professional work (the studies in educational philosophy,\n history of pedagogy and comparative educational studies); meanwhile, some\n other NAESU works (textbooks, methodological recommendations) lack a\n systematic scientific approach, because they are based on their authors\u2019\n personal beliefs or experiences or on isolated experiments which need to be\n described.\n<\/p>\n

              \n In general, the respondents in the in-depth interviews were more critical\n of the NAESU\u2019s research results and noted that they preferred foreign\n research in the field of education. In addition, they claimed that other\n IPPE employees do not read scientific papers because they do not have the\n time for it and because the language of these papers is a bit too\n exaggeratedly theoretical.\n<\/p>\n

              \n It is possible that this discrepancy in the evaluation of the use of NAESU\n research in the work of IPPEs is caused by the method of data collection\n (in the case of in-depth interviews, the respondents knew the interviewers\n personally and trusted them more), although another possible explanation is\n that it reflects the diversity of the respondents\u2019 experiences.\n<\/p>\n

              \n In order to learn about the use of NAESU research findings in pedagogical\n higher education institutions, we conducted in-depth interviews with their\n representatives, as well as with several representatives of high schools\n who are involved in education policy making. All the respondents who work\n at HEIs knew about the NAESU and its activities and had different stories\n to tell about their interaction with the NAESU, including the use of the\n NAESU research findings. They had varying opinions about the NAESU. Some\n (including school teachers) noted that they did not use NAESU research\n results in their teaching or did not even know about them. Instead, they\n used foreign research.\n<\/p>\n

              \n Some respondents have negative opinions about some specific aspects of\n NAESU activities or of the collaboration with it:\n<\/p>\n

              \n \u201cBut what kind of interaction? It sometimes takes ridiculous forms. For\n example, at nearly all meetings of the Scientific Council, we approve\n thesis topics. Traditionally, the highest number of theses, the highest\n number of PhD students are in educational sciences. And it\u2019s always funny\n for me that when the topic is announced, then the standard question always\n is whether the topic is coordinated. It is a unique case here in Ukraine\n when thesis topics must be coordinated with someone from the Academy of\n Educational Sciences; as far as I understand, it\u2019s done so that studies are\n not duplicated. Because there are so many of them that there is a high risk\n that there will be many theses on the same topic, so they are coordinated,\n these titles. This is probably the only coordination I know about.\u201d<\/blockquote>\n\n<\/p>\n

              \n Other respondents, on the contrary, have very positive opinions about the\n NAESU work and about the cooperation and possibility of cooperation with\n the Academy:\n<\/p>\n

              \n\n \u201cAs a teacher of courses in pedagogy, I want to say that we are very\n grateful to Vasyl Kremen for this encyclopaedia of education which we use,\n which the students use, which we have in the library. I must say that we\n feel considerable support through the textbooks\u2026 I can say with much\n appreciation about the textbook in didactics, published by Volodymyr\n Bondar, Svitlana Sysoyeva and Yevtukh. These are works whose target\n audience are professors of courses in pedagogy.\u201d<\/blockquote>\n\n<\/p>\n

              \n Some respondents also had some experience of working with the NAESU by\n publishing their papers in the NAESU\u2019s journals. One of them shared the\n following opinion:\n<\/p>\n

              \n\n \u201cI write a paper and send it to them. They reply the next day, \u201cYour paper\n has been accepted.\u201d There were never any questions or clarifications from\n reviewers, nothing. I mean, they pretend they review something. Maybe they\n do. But you understand that real reviews must be anonymous, there must be a\n system of communication between authors. But it doesn\u2019t happen. They write\n that it will be reviewed and might be rejected. I think that someone there\n takes a look at these articles, so that something completely terrible\n doesn\u2019t pass. But this is not real peer review. And external experts, as\n far as I understand, are not involved there, because all of this is\n supposed to take time, there should be a test for plagiarism and so on.\u201d<\/blockquote>\n\n<\/p>\n

              \n Some NAESU representatives simultaneously work at pedagogical universities.\n The respondents who have such colleagues talked about constant\n collaboration with the NAESU:\n<\/p>\n

              \n\n \u201cWe have systematic visits by representatives of the Academy of Educational\n Sciences, we have direct information, because two of our representatives\n are members. For example, the previous rector of the university was a\n member of the Presidium\u2026. So everything that happened at the Academy of\n Educational Sciences and all the decisions by the Presidium, all the\n research results, we received direct information about them at our meetings\n of Scientific Councils, Rectorates and so on.\u201d<\/blockquote>\n\n<\/p>\n<\/div>\n<\/div>\n\n\n\n\n
              \n

              Conclusions<\/h2>\n

              \n The procedures for determining the topics of studies at the NAESU in\n practice are not significantly different from the nominal procedures. We\n made this conclusion after conducting interviews with NAESU\n representatives. On the other hand, according to members of the NAESU\n Presidium, there is usually no competition between research proposals,\n because there are no alternatives to compete with. Therefore, we assume\n that in order to receive funding, it is sufficient to just meet the basic\n requirements defined by the NAESU.\n<\/p>\n

              \n The number of fundamental studies (across the NAESU in general) is almost\n three times higher than the number of practical studies. Members of the\n NAESU Presidium say that the reason for this fact is that funding for\n fundamental research is more protected. Indeed, fundamental research\n projects on theoretical topics usually receive more funds from the state\n budget than practical research projects. In general, different research\n projects of NAESU institutions (even the ones with very similar topics)\n receive very different amounts of funding. The cost of a certain research\n project depends on the number and status (degree, position) of the\n researchers involved in it. However, it is difficult to explain why similar\n research projects are so wildly different in this respect and to prove that\n the involvement of a higher number of researchers with degrees and\n positions brings a higher quality of research, rather than just higher\n funding. We can conclude that the approach to the distribution of funding\n between scientific projects within the NAESU is not perfect.\n<\/p>\n

              \n The NAESU has clearly articulated procedures for the review of research\n work: the relevant Regulations require expert evaluations, regular\n reporting and presenting the final report to the Scientific Council for\n evaluation; a negative evaluation can lead to termination of the\n researcher\u2019s labor contract. These procedures only partially measure the\n practical significance and effectiveness of research, and reports on the\n implementation of research findings mostly provide only the information\n about publishing various papers related to research projects. However, the\n implementation of findings through printed materials does not demonstrate\n their scientific quality. The actual indicators of the quality of research\n appeared in the reports of NAESU institutions only in 2017.\n<\/p>\n

              \n The evaluation of NAESU papers for meeting the basic requirements for\n scientific papers demonstrated that the majority of the evaluated papers\n fail to meet at least several criteria of quality; in particular, they do\n not provide a valid analysis of other studies on the paper\u2019s topic, do not\n mention any contemporary authors from outside the former USSR, do not\n describe the research methodology, and do not use any data processed by the\n authors themselves. Nearly all of the articles were published in Ukrainian\n journals printed by NAESU institutions or pedagogical HEIs, and the few\n journals in English are not indexed by respectable citation databases.\n Therefore, the majority of papers by researchers from three NAESU institutions do not meet the international scientific standards.\n<\/p>\n

              \n Individual NAESU representatives contribute to the making of the government\n education policies by participating in working groups which develop bills,\n by submitting suggestions for the bills, and by informally collaborating\n with the MES. In addition, the NAESU provides scientific and\n scientific-methodological support for the modernization of the content of\n education, which overlaps with the function of another institution, the\n IMCE. The NAESU research does not play any significant role in educational\n policy making, and the reasons for this are both the lack of appropriate\n research (because the NAESU\u2019s scientific inquiries are mostly focused on\n teaching methods and on the history and philosophy of education) and the\n characteristics of government policy making in general: research is\n generally rarely used in policy making in Ukraine. That is, representatives\n of the MES and its institutions also share the responsibility for the\n current situation, because they show little interest in the NAESU work.\n Meanwhile, in those MES activities which are related to the implementation\n of policies, particularly to the content of education and the organization\n of educational process, the role of the NAESU is significant. However,\n taking into account the movement towards the autonomization of schools and\n the reconsideration of the MES\u2019s functions in the management of education,\n it is likely that this part of the Ministry\u2019s activities will be gradually\n cut down.\n<\/p>\n

              \n The respondents from the UCEQA and the IEA collaborate with the NAESU in\n their everyday work via individual NAESU representatives, but they do not\n use their research findings. Heads of IPPEs are critical of the NAESU\n research results and do not use them, but other IPPE representatives\n generally share positive opinions about the NAESU work.\n<\/p>\n

              \n The practice of involving external stakeholders in the discussion of\n research plans and priorities has certain shortcomings. The NAESU virtually\n never interacts with Ukrainian education experts from outside the NAESU.\n The respondents from this group usually had no knowledge of the NAESU\n research or scientific papers, and sometimes they did not even know about\n the main topics studied by institutions related to their field of\n expertise. Instead, the education experts mostly use foreign research in\n their work. They also have had no chance to suggest research topics for the\n NAESU or to participate in determining the NAESU priorities.\n<\/p>\n

              \n The representatives of MES and its subordinate institutions have uneven\n experiences of discussing the NAESU research plans. Some of them\n participate in approving these plans only formally, and some do it more\n actively. Sporadic coordination of plans and priorities mostly happens\n during informal communication between representatives of the NAESU, the\n MES, and the Ministry\u2019s institutions.\n<\/p>\n

              \n The situation is similar among the representatives of education\n institutions. Among the surveyed employees of higher education institutions\n there was no uniform opinion about the collaboration with the NAESU (if it\n exists): there were both positive and critical answers.\n<\/p>\n

              \n To sum up the findings of this study, we have reached the following\n conclusions. The scientific level of articles produced by at least three\n NAESU institutions does not meet the international standards. The\n competitions of research topics are usually non-competitive for the lack of\n alternative projects. The amount of funding for a particular research\n project is determined by the number of the researchers with degrees and\n positions who are involved in it, rather than by the priority of the\n researched topic. The results of the studies carried out by the three NAESU\n institutions are generally not used in the making and implementation of\n educational policies. Therefore, in our opinion, there is no reason to\n believe that the NAESU has the conditions for the efficient use of public\n funding.\n<\/p>\n<\/div>\n\n\n\n\n

              \n

              Recommendations Based on the Results of the Study<\/h2>\n
              \n

              Recommendations for the National Academy of Educational Sciences of Ukraine<\/h3>\n

              \n We suggest that the NAESU reconsiders its procedures for organizing\n competitions for the funding of research projects and its procedures of\n quality assurance of studies, ensures mandatory participation of external\n reviewers (unaffiliated with the NAESU) in these procedures, and tries to\n avoid any conflict of interest.\n <\/p>\n

              \n We recommend that NAESU separates its areas of responsibility (formally and\n in practice) from those of the IMCE, leaving the tasks of updating the\n content of education, creating curricula and textbooks to the IMCE in order\n to avoid functional overlap, and that the NAESU focuses more on the\n scientific research as such.\n <\/p>\n

              \n We recommend the NAESU to pay special attention to the external\n communication of its research findings and the work of its press center,\n because the better these findings are presented in the public space, the\n higher is the outreach to the consumers and the visibility of the\n institution itself.\n <\/p>\n

              \n We advise the NAESU to change the editorial policies of the NAESU\u2019s own\n journals, bringing them up to the global standards.\n <\/p>\n

              \n We recommend that the NAESU introduces formal criteria of evaluation of its\n employees\u2019 scientific papers, such as the citation index and publications\n in journals which are indexed by Scopus or the Web of Science.\n <\/p>\n <\/div>\n

              \n

              Recommendations for the National Ukrainian Council on the Development of Science and Technology<\/h3>\n

              \n We recommend the Council to define the priority topics in education which\n need to be researched.\n<\/p>\n

              \n We recommend the Council to initiate a thorough audit of the NAESU\u2019s\n scientific activities in terms of the relevance and novelty of its research\n in the context of the needs of Ukrainian education and the global\n tendencies in educational research.\n<\/p>\n

              \n We recommend to review the rules for the distribution of funding for\n research, in order to take into account the priorities, the relevance and\n the scientific quality.\n<\/p>\n

              \n We also recommend to introduce grant-based funding as the prevalent way of\n funding scientific studies, which would open the opportunity to receive the\n funding to conduct these studies for all research institutions, regardless\n of whether they are public or private, and for all teams of authors who can\n prove their capacity to provide the conditions for the effective use of the\n funding. We also recommend to involve representatives of stakeholders,\n especially the MES, in the competition commission that would consider the\n grant applications.\n<\/p><\/div>\n

              \n

              Recommendations for the Ministry of Education and Science of Ukraine<\/h3>\n

              \n We recommend to allocate resources for the development of the Ministry\u2019s\n subordinate institutions which will productively work directly on the tasks\n of updating the content of education and providing scientific and\n methodological support.\n<\/p>\n

              \n We recommend to introduce long-term planning of priorities in educational\n research. We also recommend the MES to announce open competitions of\n research projects and policy analyses in education, in order to use the\n budget money to obtain products of higher quality, which can be used in\n fulfilling the Ministry\u2019s functions.\n<\/p>\n

              \n In order to improve the quality of educational policies, we recommend that\n the MES changes the processes of policy making and the development of\n regulations, introducing the stage of data collection and analysis,\n reviewing the relevant research, and forming the requests for the data and\n studies which need to be collected and carried out.\n<\/p>\n<\/div>\n

              \n

              Recommendations for the Ministry of Finance of Ukraine<\/h3>\n

              \n When determining the amount of funding for the NAESU, we recommend the\n Ministry of Finance:\n<\/p>\n

                \n
              • \n

                \n to take into account the official position of the MES about the\n concrete practical use of any particular NAESU research project for\n the policies of the Ministry of Education and Science; in\n particular, to request from the MES the list of the specific steps\n needed to implement certain policies and individual regulations in\n which the NAESU research findings could be used;\n <\/p>\n <\/li>\n

              • \n

                \n to take into account the evaluation of the NAESU\u2019s report for the\n previous period, made by the National Ukrainian Council for Science\n and Technology.\n <\/p>\n <\/li>\n<\/ul>\n<\/div><\/div>\n

                \n

                Appendix<\/h2>\n
                  \n
                1. \n

                  \n Statute of the National Academy of Educational Sciences of Ukraine.\n <\/p>\n <\/li>\n

                2. \n

                  \n Strategy of the Development of the National Academy of Educational\n Sciences of Ukraine for 2016-22.\n <\/p>\n <\/li>\n

                3. \n

                  \n Report on the Activities of the National Academy of Educational\n Sciences of Ukraine in 2014.\n <\/p>\n <\/li>\n

                4. \n

                  \n Report on the Activities of the National Academy of Educational\n Sciences of Ukraine in 2015.\n <\/p>\n <\/li>\n

                5. \n

                  \n Report on the Activities of the National Academy of Educational\n Sciences of Ukraine in 2016 and for the 2012-16 period.\n <\/p>\n <\/li>\n

                6. \n

                  \n The NAESU\u2019s information on the implementation of the State Budget\n of Ukraine in 2014 (archive from the website).\n <\/p>\n <\/li>\n

                7. \n

                  \n The NAESU\u2019s information on the implementation of the State Budget\n of Ukraine in 2015 (archive from the website).\n <\/p>\n <\/li>\n

                8. \n

                  \n The NAESU\u2019s information on the implementation of the State Budget\n of Ukraine in 2016 (archive from the website).\n <\/p>\n <\/li>\n

                9. \n

                  \n Program of Collaboration between the Ministry of Education and\n Science of Ukraine and the National Academy of Educational Sciences\n of Ukraine for 2014-16.\n <\/p>\n <\/li>\n

                10. \n

                  \n Program of Collaboration between the Ministry of Education and\n Science of Ukraine and the National Academy of Educational Sciences\n of Ukraine for 2017-20.\n <\/p>\n <\/li>\n

                11. \n

                  \n Regulations on the Procedure of Planning and Implementation of\n Research Projects at the National Academy of Educational Sciences\n of Ukraine for 2011 (with changes made n 2012, 2014, 2016 and\n 2017).\n <\/p>\n <\/li>\n

                12. \n

                  \n Report on Implementing the Passport of the Budget Program as of\n January 1, 2015 (received upon request).\n <\/p>\n <\/li>\n

                13. \n

                  \n Report on Implementing the Passport of the Budget Program as of\n January 1, 2016 (received upon request).\n <\/p>\n <\/li>\n

                14. \n

                  \n Report on Implementing the Passport of the Budget Program as of\n January 1, 2017 (received upon request).\n <\/p>\n <\/li>\n

                15. \n

                  \n Decree of the Presidium “On announcing the competition of research\n projects on sectoral topics whose implementation will start in\n 2016\u201d, January 22, 2015 (received upon request).\n <\/p>\n <\/li>\n

                16. \n

                  \n Decree of the Presidium “On the results of the competition of\n research projects on sectoral topics of the NAES of Ukraine, whose\n implementation will start in 2016,\u201d April 1, 2015 (received upon\n request).\n <\/p>\n <\/li>\n

                17. \n

                  \n Decree of the Presidium “On announcing the competition of research\n projects on sectoral topics whose implementation will start in\n 2017\u201d, January 5, 2016 (received upon request).\n <\/p>\n <\/li>\n

                18. \n

                  \n Decree of the Presidium “On announcing the competition of research\n projects on sectoral topics whose implementation will start in\n 2016\u201d, April 22, 2016 (received upon request).\n <\/p>\n <\/li>\n

                19. \n

                  \n Decree of the Presidium “On the results of the competition of\n research projects on sectoral topics of the NAES of Ukraine, whose\n implementation will start in 2017,\u201d February 23, 2016 (received\n upon request).\n <\/p>\n <\/li>\n

                20. \n

                  \n Decree of the Presidium “On the results of the competition of\n research projects on sectoral topics of the NAES of Ukraine, whose\n implementation will start in 2017,\u201d April 21, 2016 (received upon\n request).\n <\/p>\n <\/li>\n

                21. \n

                  \n The list of scientific publications by the Institute of Higher\n Education in 2014, 2015 and 2016 (received upon request).\n <\/p>\n <\/li>\n

                22. \n

                  \n The thematic plans of practical and fundamental research projects\n of the National Academy of Educational Sciences of Ukraine for\n 2013, 2014, 2015, 2016 and 2017 (received upon request).\n <\/p>\n <\/li>\n

                23. \n

                  \n Staffing tables of the NAESU institutions in 2013, 2014, 2015, 2016\n and 2017 (received upon request).\n <\/p>\n <\/li>\n<\/ol>\n <\/div>\n\n","protected":false},"excerpt":{"rendered":"

                  Many countries constantly search for ways to spend public budget funds more efficiently. Our study aims to determine if the procedures and practices of the planning and organization of research at the NAESU can ensure the efficient use of public funding in its scientific activities.<\/p>\n","protected":false},"featured_media":0,"parent":0,"menu_order":0,"template":"","topic":[14],"acf":[],"_links":{"self":[{"href":"https:\/\/cedos.org.ua\/en\/wp-json\/wp\/v2\/researches\/3764"}],"collection":[{"href":"https:\/\/cedos.org.ua\/en\/wp-json\/wp\/v2\/researches"}],"about":[{"href":"https:\/\/cedos.org.ua\/en\/wp-json\/wp\/v2\/types\/researches"}],"wp:attachment":[{"href":"https:\/\/cedos.org.ua\/en\/wp-json\/wp\/v2\/media?parent=3764"}],"wp:term":[{"taxonomy":"topic","embeddable":true,"href":"https:\/\/cedos.org.ua\/en\/wp-json\/wp\/v2\/topic?post=3764"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}